2023-05-10 04:53:32,883 INFO [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure timeout: 13 mins 2023-05-10 04:53:33,166 DEBUG [main] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5d73dea0-99f3-3103-481a-a34861abc6a4 2023-05-10 04:53:33,181 INFO [Time-limited test] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=3, rsPorts=, rsClass=null, numDataNodes=3, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-05-10 04:53:33,182 INFO [Time-limited test] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5d73dea0-99f3-3103-481a-a34861abc6a4/cluster_105d7649-9bd1-85f0-42ad-07d3b5dd11a5, deleteOnExit=true 2023-05-10 04:53:33,182 INFO [Time-limited test] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-05-10 04:53:33,183 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5d73dea0-99f3-3103-481a-a34861abc6a4/test.cache.data in system properties and HBase conf 2023-05-10 04:53:33,183 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5d73dea0-99f3-3103-481a-a34861abc6a4/hadoop.tmp.dir in system properties and HBase conf 2023-05-10 04:53:33,184 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5d73dea0-99f3-3103-481a-a34861abc6a4/hadoop.log.dir in system properties and HBase conf 2023-05-10 04:53:33,184 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5d73dea0-99f3-3103-481a-a34861abc6a4/mapreduce.cluster.local.dir in system properties and HBase conf 2023-05-10 04:53:33,185 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5d73dea0-99f3-3103-481a-a34861abc6a4/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-05-10 04:53:33,185 INFO [Time-limited test] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-05-10 04:53:33,310 WARN [Time-limited test] util.NativeCodeLoader(62): Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2023-05-10 04:53:33,765 DEBUG [Time-limited test] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-05-10 04:53:33,771 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5d73dea0-99f3-3103-481a-a34861abc6a4/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-05-10 04:53:33,772 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5d73dea0-99f3-3103-481a-a34861abc6a4/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-05-10 04:53:33,772 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5d73dea0-99f3-3103-481a-a34861abc6a4/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-05-10 04:53:33,773 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5d73dea0-99f3-3103-481a-a34861abc6a4/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-05-10 04:53:33,773 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5d73dea0-99f3-3103-481a-a34861abc6a4/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-05-10 04:53:33,774 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5d73dea0-99f3-3103-481a-a34861abc6a4/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-05-10 04:53:33,774 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5d73dea0-99f3-3103-481a-a34861abc6a4/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-05-10 04:53:33,774 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5d73dea0-99f3-3103-481a-a34861abc6a4/dfs.journalnode.edits.dir in system properties and HBase conf 2023-05-10 04:53:33,775 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5d73dea0-99f3-3103-481a-a34861abc6a4/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-05-10 04:53:33,775 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5d73dea0-99f3-3103-481a-a34861abc6a4/nfs.dump.dir in system properties and HBase conf 2023-05-10 04:53:33,776 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5d73dea0-99f3-3103-481a-a34861abc6a4/java.io.tmpdir in system properties and HBase conf 2023-05-10 04:53:33,776 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5d73dea0-99f3-3103-481a-a34861abc6a4/dfs.journalnode.edits.dir in system properties and HBase conf 2023-05-10 04:53:33,777 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5d73dea0-99f3-3103-481a-a34861abc6a4/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-05-10 04:53:33,777 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5d73dea0-99f3-3103-481a-a34861abc6a4/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-05-10 04:53:34,451 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-05-10 04:53:34,457 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-05-10 04:53:34,827 WARN [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties 2023-05-10 04:53:35,020 INFO [Time-limited test] log.Slf4jLog(67): Logging to org.slf4j.impl.Reload4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2023-05-10 04:53:35,035 WARN [Time-limited test] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-05-10 04:53:35,087 INFO [Time-limited test] log.Slf4jLog(67): jetty-6.1.26 2023-05-10 04:53:35,123 INFO [Time-limited test] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5d73dea0-99f3-3103-481a-a34861abc6a4/java.io.tmpdir/Jetty_localhost_46623_hdfs____.412o81/webapp 2023-05-10 04:53:35,270 INFO [Time-limited test] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:46623 2023-05-10 04:53:35,288 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-05-10 04:53:35,289 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-05-10 04:53:35,884 WARN [Listener at localhost/38753] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-05-10 04:53:35,971 WARN [Listener at localhost/38753] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-05-10 04:53:35,991 WARN [Listener at localhost/38753] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-05-10 04:53:35,998 INFO [Listener at localhost/38753] log.Slf4jLog(67): jetty-6.1.26 2023-05-10 04:53:36,003 INFO [Listener at localhost/38753] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5d73dea0-99f3-3103-481a-a34861abc6a4/java.io.tmpdir/Jetty_localhost_45923_datanode____lyz62o/webapp 2023-05-10 04:53:36,103 INFO [Listener at localhost/38753] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:45923 2023-05-10 04:53:36,473 WARN [Listener at localhost/33201] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-05-10 04:53:36,534 WARN [Listener at localhost/33201] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-05-10 04:53:36,538 WARN [Listener at localhost/33201] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-05-10 04:53:36,540 INFO [Listener at localhost/33201] log.Slf4jLog(67): jetty-6.1.26 2023-05-10 04:53:36,556 INFO [Listener at localhost/33201] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5d73dea0-99f3-3103-481a-a34861abc6a4/java.io.tmpdir/Jetty_localhost_33853_datanode____.xjdm03/webapp 2023-05-10 04:53:36,703 INFO [Listener at localhost/33201] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:33853 2023-05-10 04:53:36,716 WARN [Listener at localhost/39309] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-05-10 04:53:36,749 WARN [Listener at localhost/39309] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-05-10 04:53:36,753 WARN [Listener at localhost/39309] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-05-10 04:53:36,755 INFO [Listener at localhost/39309] log.Slf4jLog(67): jetty-6.1.26 2023-05-10 04:53:36,766 INFO [Listener at localhost/39309] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5d73dea0-99f3-3103-481a-a34861abc6a4/java.io.tmpdir/Jetty_localhost_33933_datanode____8467hc/webapp 2023-05-10 04:53:36,899 INFO [Listener at localhost/39309] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:33933 2023-05-10 04:53:36,914 WARN [Listener at localhost/45007] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-05-10 04:53:37,129 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xf956a3c5e6467340: Processing first storage report for DS-a3c93b79-41c5-4541-ba4f-e18855846a32 from datanode 87e3fd95-9de7-4a9c-ae21-87bf81b5e4b4 2023-05-10 04:53:37,130 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xf956a3c5e6467340: from storage DS-a3c93b79-41c5-4541-ba4f-e18855846a32 node DatanodeRegistration(127.0.0.1:42419, datanodeUuid=87e3fd95-9de7-4a9c-ae21-87bf81b5e4b4, infoPort=44343, infoSecurePort=0, ipcPort=33201, storageInfo=lv=-57;cid=testClusterID;nsid=1751125290;c=1683694414559), blocks: 0, hasStaleStorage: true, processing time: 1 msecs, invalidatedBlocks: 0 2023-05-10 04:53:37,131 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x663ec7f38e67fb2e: Processing first storage report for DS-6547f095-4b0d-4558-8c35-c9b2bf842d27 from datanode 0bc2fe53-c3db-437e-a8da-2e5e8eeaaf20 2023-05-10 04:53:37,131 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x663ec7f38e67fb2e: from storage DS-6547f095-4b0d-4558-8c35-c9b2bf842d27 node DatanodeRegistration(127.0.0.1:43517, datanodeUuid=0bc2fe53-c3db-437e-a8da-2e5e8eeaaf20, infoPort=38919, infoSecurePort=0, ipcPort=45007, storageInfo=lv=-57;cid=testClusterID;nsid=1751125290;c=1683694414559), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-05-10 04:53:37,131 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x9f614def84f444f6: Processing first storage report for DS-a5b310f7-364e-4271-a65f-70861aa844d7 from datanode a7b70f5c-571f-439d-b854-0d142c7d1170 2023-05-10 04:53:37,131 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x9f614def84f444f6: from storage DS-a5b310f7-364e-4271-a65f-70861aa844d7 node DatanodeRegistration(127.0.0.1:41547, datanodeUuid=a7b70f5c-571f-439d-b854-0d142c7d1170, infoPort=38147, infoSecurePort=0, ipcPort=39309, storageInfo=lv=-57;cid=testClusterID;nsid=1751125290;c=1683694414559), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-05-10 04:53:37,131 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0xf956a3c5e6467340: Processing first storage report for DS-d82787bf-3076-41e4-b3d5-130d08f0cd30 from datanode 87e3fd95-9de7-4a9c-ae21-87bf81b5e4b4 2023-05-10 04:53:37,131 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0xf956a3c5e6467340: from storage DS-d82787bf-3076-41e4-b3d5-130d08f0cd30 node DatanodeRegistration(127.0.0.1:42419, datanodeUuid=87e3fd95-9de7-4a9c-ae21-87bf81b5e4b4, infoPort=44343, infoSecurePort=0, ipcPort=33201, storageInfo=lv=-57;cid=testClusterID;nsid=1751125290;c=1683694414559), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-05-10 04:53:37,132 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x663ec7f38e67fb2e: Processing first storage report for DS-7d0a9b8b-e4f3-4e7a-ab18-91679ae72f5b from datanode 0bc2fe53-c3db-437e-a8da-2e5e8eeaaf20 2023-05-10 04:53:37,132 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x663ec7f38e67fb2e: from storage DS-7d0a9b8b-e4f3-4e7a-ab18-91679ae72f5b node DatanodeRegistration(127.0.0.1:43517, datanodeUuid=0bc2fe53-c3db-437e-a8da-2e5e8eeaaf20, infoPort=38919, infoSecurePort=0, ipcPort=45007, storageInfo=lv=-57;cid=testClusterID;nsid=1751125290;c=1683694414559), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-05-10 04:53:37,132 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x9f614def84f444f6: Processing first storage report for DS-fb963941-8642-45f0-a043-ff3c15a0423e from datanode a7b70f5c-571f-439d-b854-0d142c7d1170 2023-05-10 04:53:37,132 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x9f614def84f444f6: from storage DS-fb963941-8642-45f0-a043-ff3c15a0423e node DatanodeRegistration(127.0.0.1:41547, datanodeUuid=a7b70f5c-571f-439d-b854-0d142c7d1170, infoPort=38147, infoSecurePort=0, ipcPort=39309, storageInfo=lv=-57;cid=testClusterID;nsid=1751125290;c=1683694414559), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-05-10 04:53:37,386 DEBUG [Listener at localhost/45007] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5d73dea0-99f3-3103-481a-a34861abc6a4 2023-05-10 04:53:37,449 INFO [Listener at localhost/45007] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5d73dea0-99f3-3103-481a-a34861abc6a4/cluster_105d7649-9bd1-85f0-42ad-07d3b5dd11a5/zookeeper_0, clientPort=51177, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5d73dea0-99f3-3103-481a-a34861abc6a4/cluster_105d7649-9bd1-85f0-42ad-07d3b5dd11a5/zookeeper_0/version-2, dataDirSize=424 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5d73dea0-99f3-3103-481a-a34861abc6a4/cluster_105d7649-9bd1-85f0-42ad-07d3b5dd11a5/zookeeper_0/version-2, dataLogSize=424 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, serverId=0 2023-05-10 04:53:37,463 INFO [Listener at localhost/45007] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=51177 2023-05-10 04:53:37,471 INFO [Listener at localhost/45007] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-05-10 04:53:37,474 INFO [Listener at localhost/45007] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-05-10 04:53:38,186 INFO [Listener at localhost/45007] util.FSUtils(471): Created version file at hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7 with version=8 2023-05-10 04:53:38,187 INFO [Listener at localhost/45007] hbase.HBaseTestingUtility(1406): Setting hbase.fs.tmp.dir to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/hbase-staging 2023-05-10 04:53:38,492 INFO [Listener at localhost/45007] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl 2023-05-10 04:53:38,932 INFO [Listener at localhost/45007] client.ConnectionUtils(127): master/jenkins-hbase4:0 server-side Connection retries=45 2023-05-10 04:53:38,964 INFO [Listener at localhost/45007] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-05-10 04:53:38,964 INFO [Listener at localhost/45007] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-05-10 04:53:38,965 INFO [Listener at localhost/45007] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-05-10 04:53:38,965 INFO [Listener at localhost/45007] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-05-10 04:53:38,965 INFO [Listener at localhost/45007] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-05-10 04:53:39,102 INFO [Listener at localhost/45007] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-05-10 04:53:39,172 DEBUG [Listener at localhost/45007] util.ClassSize(228): Using Unsafe to estimate memory layout 2023-05-10 04:53:39,263 INFO [Listener at localhost/45007] ipc.NettyRpcServer(120): Bind to /172.31.14.131:37903 2023-05-10 04:53:39,273 INFO [Listener at localhost/45007] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-05-10 04:53:39,275 INFO [Listener at localhost/45007] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-05-10 04:53:39,295 INFO [Listener at localhost/45007] zookeeper.RecoverableZooKeeper(93): Process identifier=master:37903 connecting to ZooKeeper ensemble=127.0.0.1:51177 2023-05-10 04:53:39,338 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:379030x0, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-05-10 04:53:39,340 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): master:37903-0x1001370791d0000 connected 2023-05-10 04:53:39,367 DEBUG [Listener at localhost/45007] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-05-10 04:53:39,368 DEBUG [Listener at localhost/45007] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-05-10 04:53:39,371 DEBUG [Listener at localhost/45007] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-05-10 04:53:39,379 DEBUG [Listener at localhost/45007] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=37903 2023-05-10 04:53:39,379 DEBUG [Listener at localhost/45007] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=37903 2023-05-10 04:53:39,380 DEBUG [Listener at localhost/45007] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=37903 2023-05-10 04:53:39,380 DEBUG [Listener at localhost/45007] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=37903 2023-05-10 04:53:39,380 DEBUG [Listener at localhost/45007] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=37903 2023-05-10 04:53:39,386 INFO [Listener at localhost/45007] master.HMaster(444): hbase.rootdir=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7, hbase.cluster.distributed=false 2023-05-10 04:53:39,452 INFO [Listener at localhost/45007] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-05-10 04:53:39,453 INFO [Listener at localhost/45007] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-05-10 04:53:39,453 INFO [Listener at localhost/45007] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-05-10 04:53:39,453 INFO [Listener at localhost/45007] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-05-10 04:53:39,453 INFO [Listener at localhost/45007] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-05-10 04:53:39,453 INFO [Listener at localhost/45007] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-05-10 04:53:39,458 INFO [Listener at localhost/45007] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-05-10 04:53:39,461 INFO [Listener at localhost/45007] ipc.NettyRpcServer(120): Bind to /172.31.14.131:36665 2023-05-10 04:53:39,463 INFO [Listener at localhost/45007] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-05-10 04:53:39,469 DEBUG [Listener at localhost/45007] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-05-10 04:53:39,470 INFO [Listener at localhost/45007] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-05-10 04:53:39,472 INFO [Listener at localhost/45007] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-05-10 04:53:39,473 INFO [Listener at localhost/45007] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:36665 connecting to ZooKeeper ensemble=127.0.0.1:51177 2023-05-10 04:53:39,477 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:366650x0, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-05-10 04:53:39,478 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:36665-0x1001370791d0001 connected 2023-05-10 04:53:39,478 DEBUG [Listener at localhost/45007] zookeeper.ZKUtil(164): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-05-10 04:53:39,479 DEBUG [Listener at localhost/45007] zookeeper.ZKUtil(164): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-05-10 04:53:39,480 DEBUG [Listener at localhost/45007] zookeeper.ZKUtil(164): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-05-10 04:53:39,481 DEBUG [Listener at localhost/45007] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=36665 2023-05-10 04:53:39,481 DEBUG [Listener at localhost/45007] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=36665 2023-05-10 04:53:39,483 DEBUG [Listener at localhost/45007] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=36665 2023-05-10 04:53:39,486 DEBUG [Listener at localhost/45007] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=36665 2023-05-10 04:53:39,487 DEBUG [Listener at localhost/45007] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=36665 2023-05-10 04:53:39,499 INFO [Listener at localhost/45007] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-05-10 04:53:39,499 INFO [Listener at localhost/45007] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-05-10 04:53:39,499 INFO [Listener at localhost/45007] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-05-10 04:53:39,500 INFO [Listener at localhost/45007] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-05-10 04:53:39,500 INFO [Listener at localhost/45007] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-05-10 04:53:39,500 INFO [Listener at localhost/45007] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-05-10 04:53:39,500 INFO [Listener at localhost/45007] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-05-10 04:53:39,502 INFO [Listener at localhost/45007] ipc.NettyRpcServer(120): Bind to /172.31.14.131:37995 2023-05-10 04:53:39,502 INFO [Listener at localhost/45007] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-05-10 04:53:39,503 DEBUG [Listener at localhost/45007] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-05-10 04:53:39,504 INFO [Listener at localhost/45007] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-05-10 04:53:39,505 INFO [Listener at localhost/45007] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-05-10 04:53:39,507 INFO [Listener at localhost/45007] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:37995 connecting to ZooKeeper ensemble=127.0.0.1:51177 2023-05-10 04:53:39,514 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:379950x0, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-05-10 04:53:39,515 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:37995-0x1001370791d0002 connected 2023-05-10 04:53:39,515 DEBUG [Listener at localhost/45007] zookeeper.ZKUtil(164): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-05-10 04:53:39,515 DEBUG [Listener at localhost/45007] zookeeper.ZKUtil(164): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-05-10 04:53:39,516 DEBUG [Listener at localhost/45007] zookeeper.ZKUtil(164): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-05-10 04:53:39,519 DEBUG [Listener at localhost/45007] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=37995 2023-05-10 04:53:39,519 DEBUG [Listener at localhost/45007] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=37995 2023-05-10 04:53:39,519 DEBUG [Listener at localhost/45007] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=37995 2023-05-10 04:53:39,520 DEBUG [Listener at localhost/45007] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=37995 2023-05-10 04:53:39,522 DEBUG [Listener at localhost/45007] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=37995 2023-05-10 04:53:39,535 INFO [Listener at localhost/45007] client.ConnectionUtils(127): regionserver/jenkins-hbase4:0 server-side Connection retries=45 2023-05-10 04:53:39,535 INFO [Listener at localhost/45007] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-05-10 04:53:39,535 INFO [Listener at localhost/45007] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-05-10 04:53:39,535 INFO [Listener at localhost/45007] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-05-10 04:53:39,535 INFO [Listener at localhost/45007] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-05-10 04:53:39,535 INFO [Listener at localhost/45007] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-05-10 04:53:39,535 INFO [Listener at localhost/45007] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-05-10 04:53:39,537 INFO [Listener at localhost/45007] ipc.NettyRpcServer(120): Bind to /172.31.14.131:37823 2023-05-10 04:53:39,538 INFO [Listener at localhost/45007] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-05-10 04:53:39,542 DEBUG [Listener at localhost/45007] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-05-10 04:53:39,544 INFO [Listener at localhost/45007] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-05-10 04:53:39,546 INFO [Listener at localhost/45007] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-05-10 04:53:39,547 INFO [Listener at localhost/45007] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:37823 connecting to ZooKeeper ensemble=127.0.0.1:51177 2023-05-10 04:53:39,551 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:378230x0, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-05-10 04:53:39,552 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:37823-0x1001370791d0003 connected 2023-05-10 04:53:39,552 DEBUG [Listener at localhost/45007] zookeeper.ZKUtil(164): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-05-10 04:53:39,553 DEBUG [Listener at localhost/45007] zookeeper.ZKUtil(164): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-05-10 04:53:39,554 DEBUG [Listener at localhost/45007] zookeeper.ZKUtil(164): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-05-10 04:53:39,555 DEBUG [Listener at localhost/45007] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=37823 2023-05-10 04:53:39,555 DEBUG [Listener at localhost/45007] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=37823 2023-05-10 04:53:39,555 DEBUG [Listener at localhost/45007] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=37823 2023-05-10 04:53:39,556 DEBUG [Listener at localhost/45007] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=37823 2023-05-10 04:53:39,556 DEBUG [Listener at localhost/45007] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=37823 2023-05-10 04:53:39,563 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /hbase/backup-masters/jenkins-hbase4.apache.org,37903,1683694418326 2023-05-10 04:53:39,572 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-05-10 04:53:39,574 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/jenkins-hbase4.apache.org,37903,1683694418326 2023-05-10 04:53:39,592 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-05-10 04:53:39,592 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-05-10 04:53:39,592 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-05-10 04:53:39,592 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-05-10 04:53:39,593 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-05-10 04:53:39,594 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-05-10 04:53:39,596 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-05-10 04:53:39,596 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /hbase/backup-masters/jenkins-hbase4.apache.org,37903,1683694418326 from backup master directory 2023-05-10 04:53:39,600 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/jenkins-hbase4.apache.org,37903,1683694418326 2023-05-10 04:53:39,600 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-05-10 04:53:39,601 WARN [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-05-10 04:53:39,601 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase4.apache.org,37903,1683694418326 2023-05-10 04:53:39,604 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating data MemStoreChunkPool with chunk size 2 MB, max count 352, initial count 0 2023-05-10 04:53:39,606 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating index MemStoreChunkPool with chunk size 204.80 KB, max count 391, initial count 0 2023-05-10 04:53:39,702 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] util.FSUtils(620): Created cluster ID file at hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/hbase.id with ID: 2113416b-2c4b-4123-a713-4efa9709895b 2023-05-10 04:53:39,749 INFO [master/jenkins-hbase4:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-05-10 04:53:39,766 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-05-10 04:53:39,815 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x28c3bf4b to 127.0.0.1:51177 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-05-10 04:53:39,849 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@3f12041e, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-05-10 04:53:39,872 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-05-10 04:53:39,874 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-05-10 04:53:39,891 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(264): ClientProtocol::create wrong number of arguments, should be hadoop 3.2 or below 2023-05-10 04:53:39,891 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(270): ClientProtocol::create wrong number of arguments, should be hadoop 2.x 2023-05-10 04:53:39,894 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(279): can not find SHOULD_REPLICATE flag, should be hadoop 2.x java.lang.IllegalArgumentException: No enum constant org.apache.hadoop.fs.CreateFlag.SHOULD_REPLICATE at java.lang.Enum.valueOf(Enum.java:238) at org.apache.hadoop.fs.CreateFlag.valueOf(CreateFlag.java:63) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.loadShouldReplicateFlag(FanOutOneBlockAsyncDFSOutputHelper.java:277) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.(FanOutOneBlockAsyncDFSOutputHelper.java:304) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:139) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:855) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2193) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:528) at java.lang.Thread.run(Thread.java:750) 2023-05-10 04:53:39,899 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(243): No decryptEncryptedDataEncryptionKey method in DFSClient, should be hadoop version with HDFS-12396 java.lang.NoSuchMethodException: org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(org.apache.hadoop.fs.FileEncryptionInfo) at java.lang.Class.getDeclaredMethod(Class.java:2130) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelperWithoutHDFS12396(FanOutOneBlockAsyncDFSOutputSaslHelper.java:182) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelper(FanOutOneBlockAsyncDFSOutputSaslHelper.java:241) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.(FanOutOneBlockAsyncDFSOutputSaslHelper.java:252) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:140) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:855) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2193) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:528) at java.lang.Thread.run(Thread.java:750) 2023-05-10 04:53:39,901 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-05-10 04:53:39,936 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7693): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/MasterData/data/master/store-tmp 2023-05-10 04:53:40,005 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-10 04:53:40,005 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-05-10 04:53:40,005 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-05-10 04:53:40,005 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-05-10 04:53:40,005 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-05-10 04:53:40,005 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-05-10 04:53:40,006 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-05-10 04:53:40,006 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-05-10 04:53:40,007 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/MasterData/WALs/jenkins-hbase4.apache.org,37903,1683694418326 2023-05-10 04:53:40,026 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C37903%2C1683694418326, suffix=, logDir=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/MasterData/WALs/jenkins-hbase4.apache.org,37903,1683694418326, archiveDir=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/MasterData/oldWALs, maxLogs=10 2023-05-10 04:53:40,082 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43517,DS-6547f095-4b0d-4558-8c35-c9b2bf842d27,DISK] 2023-05-10 04:53:40,082 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:42419,DS-a3c93b79-41c5-4541-ba4f-e18855846a32,DISK] 2023-05-10 04:53:40,082 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:41547,DS-a5b310f7-364e-4271-a65f-70861aa844d7,DISK] 2023-05-10 04:53:40,090 DEBUG [RS-EventLoopGroup-5-2] asyncfs.ProtobufDecoder(123): Hadoop 3.2 and below use unshaded protobuf. java.lang.ClassNotFoundException: org.apache.hadoop.thirdparty.protobuf.MessageLite at java.net.URLClassLoader.findClass(URLClassLoader.java:387) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.io.asyncfs.ProtobufDecoder.(ProtobufDecoder.java:118) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.processWriteBlockResponse(FanOutOneBlockAsyncDFSOutputHelper.java:340) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$100(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$4.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:424) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:557) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.addListener(DefaultPromise.java:185) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:418) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:476) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:471) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:625) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:105) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:84) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:653) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:691) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-05-10 04:53:40,170 INFO [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/MasterData/WALs/jenkins-hbase4.apache.org,37903,1683694418326/jenkins-hbase4.apache.org%2C37903%2C1683694418326.1683694420036 2023-05-10 04:53:40,170 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:41547,DS-a5b310f7-364e-4271-a65f-70861aa844d7,DISK], DatanodeInfoWithStorage[127.0.0.1:43517,DS-6547f095-4b0d-4558-8c35-c9b2bf842d27,DISK], DatanodeInfoWithStorage[127.0.0.1:42419,DS-a3c93b79-41c5-4541-ba4f-e18855846a32,DISK]] 2023-05-10 04:53:40,171 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-05-10 04:53:40,172 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-10 04:53:40,176 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-05-10 04:53:40,177 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-05-10 04:53:40,238 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-05-10 04:53:40,244 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-05-10 04:53:40,268 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-05-10 04:53:40,279 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-10 04:53:40,286 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-05-10 04:53:40,288 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-05-10 04:53:40,303 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-05-10 04:53:40,308 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-05-10 04:53:40,309 INFO [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62164077, jitterRate=-0.07368306815624237}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-05-10 04:53:40,309 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-05-10 04:53:40,310 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-05-10 04:53:40,330 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-10 04:53:40,330 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-10 04:53:40,333 INFO [master/jenkins-hbase4:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-10 04:53:40,335 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 1 msec 2023-05-10 04:53:40,367 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 32 msec 2023-05-10 04:53:40,367 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(95): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-05-10 04:53:40,392 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-05-10 04:53:40,397 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-05-10 04:53:40,420 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-05-10 04:53:40,424 INFO [master/jenkins-hbase4:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-05-10 04:53:40,425 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/balancer 2023-05-10 04:53:40,430 INFO [master/jenkins-hbase4:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-05-10 04:53:40,434 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2023-05-10 04:53:40,437 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-05-10 04:53:40,439 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2023-05-10 04:53:40,439 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2023-05-10 04:53:40,451 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/snapshot-cleanup 2023-05-10 04:53:40,457 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-05-10 04:53:40,458 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-05-10 04:53:40,458 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-05-10 04:53:40,458 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-05-10 04:53:40,458 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-05-10 04:53:40,458 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase4.apache.org,37903,1683694418326, sessionid=0x1001370791d0000, setting cluster-up flag (Was=false) 2023-05-10 04:53:40,475 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-05-10 04:53:40,480 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/flush-table-proc/acquired, /hbase/flush-table-proc/reached, /hbase/flush-table-proc/abort 2023-05-10 04:53:40,481 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,37903,1683694418326 2023-05-10 04:53:40,491 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-05-10 04:53:40,497 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/online-snapshot/acquired, /hbase/online-snapshot/reached, /hbase/online-snapshot/abort 2023-05-10 04:53:40,499 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase4.apache.org,37903,1683694418326 2023-05-10 04:53:40,502 WARN [master/jenkins-hbase4:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.hbase-snapshot/.tmp 2023-05-10 04:53:40,562 INFO [RS:2;jenkins-hbase4:37823] regionserver.HRegionServer(951): ClusterId : 2113416b-2c4b-4123-a713-4efa9709895b 2023-05-10 04:53:40,563 INFO [RS:1;jenkins-hbase4:37995] regionserver.HRegionServer(951): ClusterId : 2113416b-2c4b-4123-a713-4efa9709895b 2023-05-10 04:53:40,562 INFO [RS:0;jenkins-hbase4:36665] regionserver.HRegionServer(951): ClusterId : 2113416b-2c4b-4123-a713-4efa9709895b 2023-05-10 04:53:40,568 DEBUG [RS:2;jenkins-hbase4:37823] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-05-10 04:53:40,568 DEBUG [RS:0;jenkins-hbase4:36665] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-05-10 04:53:40,568 DEBUG [RS:1;jenkins-hbase4:37995] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-05-10 04:53:40,575 DEBUG [RS:0;jenkins-hbase4:36665] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-05-10 04:53:40,575 DEBUG [RS:1;jenkins-hbase4:37995] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-05-10 04:53:40,575 DEBUG [RS:2;jenkins-hbase4:37823] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-05-10 04:53:40,576 DEBUG [RS:1;jenkins-hbase4:37995] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-05-10 04:53:40,575 DEBUG [RS:0;jenkins-hbase4:36665] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-05-10 04:53:40,576 DEBUG [RS:2;jenkins-hbase4:37823] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-05-10 04:53:40,579 DEBUG [RS:1;jenkins-hbase4:37995] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-05-10 04:53:40,579 DEBUG [RS:0;jenkins-hbase4:36665] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-05-10 04:53:40,579 DEBUG [RS:2;jenkins-hbase4:37823] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-05-10 04:53:40,585 DEBUG [RS:0;jenkins-hbase4:36665] zookeeper.ReadOnlyZKClient(139): Connect 0x48577ed6 to 127.0.0.1:51177 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-05-10 04:53:40,585 DEBUG [RS:1;jenkins-hbase4:37995] zookeeper.ReadOnlyZKClient(139): Connect 0x2d3688e7 to 127.0.0.1:51177 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-05-10 04:53:40,585 DEBUG [RS:2;jenkins-hbase4:37823] zookeeper.ReadOnlyZKClient(139): Connect 0x38eee696 to 127.0.0.1:51177 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-05-10 04:53:40,599 DEBUG [RS:2;jenkins-hbase4:37823] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@36291ffb, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-05-10 04:53:40,599 DEBUG [RS:2;jenkins-hbase4:37823] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@32168de6, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-05-10 04:53:40,605 DEBUG [RS:0;jenkins-hbase4:36665] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@469089ac, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-05-10 04:53:40,605 DEBUG [RS:1;jenkins-hbase4:37995] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@23f16331, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-05-10 04:53:40,605 DEBUG [RS:0;jenkins-hbase4:36665] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@42af15a6, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-05-10 04:53:40,605 DEBUG [RS:1;jenkins-hbase4:37995] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@27cee397, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-05-10 04:53:40,627 DEBUG [RS:2;jenkins-hbase4:37823] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:2;jenkins-hbase4:37823 2023-05-10 04:53:40,628 DEBUG [RS:1;jenkins-hbase4:37995] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:1;jenkins-hbase4:37995 2023-05-10 04:53:40,629 DEBUG [RS:0;jenkins-hbase4:36665] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase4:36665 2023-05-10 04:53:40,632 INFO [RS:2;jenkins-hbase4:37823] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-05-10 04:53:40,632 INFO [RS:1;jenkins-hbase4:37995] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-05-10 04:53:40,632 INFO [RS:0;jenkins-hbase4:36665] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-05-10 04:53:40,633 INFO [RS:0;jenkins-hbase4:36665] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-05-10 04:53:40,632 INFO [RS:2;jenkins-hbase4:37823] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-05-10 04:53:40,633 DEBUG [RS:0;jenkins-hbase4:36665] regionserver.HRegionServer(1022): About to register with Master. 2023-05-10 04:53:40,633 INFO [RS:1;jenkins-hbase4:37995] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-05-10 04:53:40,633 DEBUG [RS:2;jenkins-hbase4:37823] regionserver.HRegionServer(1022): About to register with Master. 2023-05-10 04:53:40,633 DEBUG [RS:1;jenkins-hbase4:37995] regionserver.HRegionServer(1022): About to register with Master. 2023-05-10 04:53:40,635 INFO [RS:1;jenkins-hbase4:37995] regionserver.HRegionServer(2809): reportForDuty to master=jenkins-hbase4.apache.org,37903,1683694418326 with isa=jenkins-hbase4.apache.org/172.31.14.131:37995, startcode=1683694419499 2023-05-10 04:53:40,635 INFO [RS:2;jenkins-hbase4:37823] regionserver.HRegionServer(2809): reportForDuty to master=jenkins-hbase4.apache.org,37903,1683694418326 with isa=jenkins-hbase4.apache.org/172.31.14.131:37823, startcode=1683694419534 2023-05-10 04:53:40,635 INFO [RS:0;jenkins-hbase4:36665] regionserver.HRegionServer(2809): reportForDuty to master=jenkins-hbase4.apache.org,37903,1683694418326 with isa=jenkins-hbase4.apache.org/172.31.14.131:36665, startcode=1683694419452 2023-05-10 04:53:40,637 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-05-10 04:53:40,649 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-05-10 04:53:40,649 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-05-10 04:53:40,649 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-05-10 04:53:40,649 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=5, maxPoolSize=5 2023-05-10 04:53:40,649 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase4:0, corePoolSize=10, maxPoolSize=10 2023-05-10 04:53:40,649 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-05-10 04:53:40,650 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-05-10 04:53:40,650 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-05-10 04:53:40,651 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683694450651 2023-05-10 04:53:40,652 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-05-10 04:53:40,655 DEBUG [RS:1;jenkins-hbase4:37995] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-05-10 04:53:40,655 DEBUG [RS:0;jenkins-hbase4:36665] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-05-10 04:53:40,656 DEBUG [RS:2;jenkins-hbase4:37823] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-05-10 04:53:40,660 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-05-10 04:53:40,660 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-05-10 04:53:40,664 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-05-10 04:53:40,666 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-05-10 04:53:40,671 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-05-10 04:53:40,671 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-05-10 04:53:40,672 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-05-10 04:53:40,672 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-05-10 04:53:40,679 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-05-10 04:53:40,682 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-05-10 04:53:40,684 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-05-10 04:53:40,685 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-05-10 04:53:40,692 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-05-10 04:53:40,695 INFO [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-05-10 04:53:40,697 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1683694420697,5,FailOnTimeoutGroup] 2023-05-10 04:53:40,698 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1683694420697,5,FailOnTimeoutGroup] 2023-05-10 04:53:40,698 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-05-10 04:53:40,698 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-05-10 04:53:40,700 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-05-10 04:53:40,700 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-05-10 04:53:40,733 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:42575, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=RegionServerStatusService 2023-05-10 04:53:40,733 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:40249, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.1 (auth:SIMPLE), service=RegionServerStatusService 2023-05-10 04:53:40,733 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:46315, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.2 (auth:SIMPLE), service=RegionServerStatusService 2023-05-10 04:53:40,734 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-05-10 04:53:40,736 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-05-10 04:53:40,736 INFO [PEWorker-1] regionserver.HRegion(7675): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7 2023-05-10 04:53:40,747 INFO [RpcServer.default.FPBQ.Fifo.handler=0,queue=0,port=37903] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:40,748 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37903] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:40,748 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.ServerManager(394): Registering regionserver=jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:40,766 DEBUG [PEWorker-1] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-10 04:53:40,769 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-05-10 04:53:40,772 DEBUG [RS:2;jenkins-hbase4:37823] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7 2023-05-10 04:53:40,772 DEBUG [RS:1;jenkins-hbase4:37995] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7 2023-05-10 04:53:40,772 DEBUG [RS:0;jenkins-hbase4:36665] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7 2023-05-10 04:53:40,772 DEBUG [RS:1;jenkins-hbase4:37995] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:38753 2023-05-10 04:53:40,772 DEBUG [RS:2;jenkins-hbase4:37823] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:38753 2023-05-10 04:53:40,772 DEBUG [RS:1;jenkins-hbase4:37995] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-05-10 04:53:40,772 DEBUG [RS:0;jenkins-hbase4:36665] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:38753 2023-05-10 04:53:40,772 DEBUG [RS:2;jenkins-hbase4:37823] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-05-10 04:53:40,772 DEBUG [RS:0;jenkins-hbase4:36665] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-05-10 04:53:40,773 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/hbase/meta/1588230740/info 2023-05-10 04:53:40,774 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-05-10 04:53:40,775 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-10 04:53:40,776 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-05-10 04:53:40,780 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/hbase/meta/1588230740/rep_barrier 2023-05-10 04:53:40,780 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-05-10 04:53:40,781 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-05-10 04:53:40,781 DEBUG [RS:1;jenkins-hbase4:37995] zookeeper.ZKUtil(162): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:40,782 DEBUG [RS:0;jenkins-hbase4:36665] zookeeper.ZKUtil(162): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:40,782 WARN [RS:1;jenkins-hbase4:37995] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-05-10 04:53:40,782 WARN [RS:0;jenkins-hbase4:36665] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-05-10 04:53:40,783 INFO [RS:1;jenkins-hbase4:37995] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-05-10 04:53:40,783 DEBUG [RS:1;jenkins-hbase4:37995] regionserver.HRegionServer(1946): logDir=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/WALs/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:40,783 INFO [RS:0;jenkins-hbase4:36665] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-05-10 04:53:40,784 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-10 04:53:40,784 DEBUG [RS:0;jenkins-hbase4:36665] regionserver.HRegionServer(1946): logDir=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/WALs/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:40,783 DEBUG [RS:2;jenkins-hbase4:37823] zookeeper.ZKUtil(162): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:40,784 WARN [RS:2;jenkins-hbase4:37823] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-05-10 04:53:40,784 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-05-10 04:53:40,784 INFO [RS:2;jenkins-hbase4:37823] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-05-10 04:53:40,784 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,37823,1683694419534] 2023-05-10 04:53:40,784 DEBUG [RS:2;jenkins-hbase4:37823] regionserver.HRegionServer(1946): logDir=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/WALs/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:40,784 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,36665,1683694419452] 2023-05-10 04:53:40,787 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase4.apache.org,37995,1683694419499] 2023-05-10 04:53:40,795 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/hbase/meta/1588230740/table 2023-05-10 04:53:40,796 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-05-10 04:53:40,797 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-10 04:53:40,799 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/hbase/meta/1588230740 2023-05-10 04:53:40,801 DEBUG [RS:2;jenkins-hbase4:37823] zookeeper.ZKUtil(162): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:40,801 DEBUG [RS:0;jenkins-hbase4:36665] zookeeper.ZKUtil(162): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:40,801 DEBUG [RS:1;jenkins-hbase4:37995] zookeeper.ZKUtil(162): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:40,801 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/hbase/meta/1588230740 2023-05-10 04:53:40,801 DEBUG [RS:0;jenkins-hbase4:36665] zookeeper.ZKUtil(162): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:40,801 DEBUG [RS:2;jenkins-hbase4:37823] zookeeper.ZKUtil(162): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:40,802 DEBUG [RS:1;jenkins-hbase4:37995] zookeeper.ZKUtil(162): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:40,802 DEBUG [RS:2;jenkins-hbase4:37823] zookeeper.ZKUtil(162): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:40,802 DEBUG [RS:0;jenkins-hbase4:36665] zookeeper.ZKUtil(162): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:40,803 DEBUG [RS:1;jenkins-hbase4:37995] zookeeper.ZKUtil(162): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:40,806 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-05-10 04:53:40,808 DEBUG [PEWorker-1] regionserver.HRegion(1055): writing seq id for 1588230740 2023-05-10 04:53:40,815 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-05-10 04:53:40,816 INFO [PEWorker-1] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62118726, jitterRate=-0.07435885071754456}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-05-10 04:53:40,816 DEBUG [PEWorker-1] regionserver.HRegion(965): Region open journal for 1588230740: 2023-05-10 04:53:40,816 DEBUG [PEWorker-1] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-05-10 04:53:40,816 INFO [PEWorker-1] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-05-10 04:53:40,816 DEBUG [PEWorker-1] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-05-10 04:53:40,816 DEBUG [PEWorker-1] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-05-10 04:53:40,816 DEBUG [PEWorker-1] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-05-10 04:53:40,818 INFO [PEWorker-1] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-05-10 04:53:40,818 DEBUG [PEWorker-1] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-05-10 04:53:40,821 DEBUG [RS:2;jenkins-hbase4:37823] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-05-10 04:53:40,821 DEBUG [RS:0;jenkins-hbase4:36665] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-05-10 04:53:40,821 DEBUG [RS:1;jenkins-hbase4:37995] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-05-10 04:53:40,825 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-05-10 04:53:40,825 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-05-10 04:53:40,833 INFO [RS:2;jenkins-hbase4:37823] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-05-10 04:53:40,833 INFO [RS:1;jenkins-hbase4:37995] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-05-10 04:53:40,833 INFO [RS:0;jenkins-hbase4:36665] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-05-10 04:53:40,834 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-05-10 04:53:40,850 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-05-10 04:53:40,852 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-05-10 04:53:40,859 INFO [RS:0;jenkins-hbase4:36665] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-05-10 04:53:40,859 INFO [RS:2;jenkins-hbase4:37823] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-05-10 04:53:40,859 INFO [RS:1;jenkins-hbase4:37995] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-05-10 04:53:40,863 INFO [RS:2;jenkins-hbase4:37823] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-05-10 04:53:40,863 INFO [RS:0;jenkins-hbase4:36665] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-05-10 04:53:40,863 INFO [RS:2;jenkins-hbase4:37823] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-05-10 04:53:40,863 INFO [RS:1;jenkins-hbase4:37995] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-05-10 04:53:40,863 INFO [RS:0;jenkins-hbase4:36665] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-05-10 04:53:40,864 INFO [RS:1;jenkins-hbase4:37995] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-05-10 04:53:40,865 INFO [RS:2;jenkins-hbase4:37823] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-05-10 04:53:40,865 INFO [RS:0;jenkins-hbase4:36665] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-05-10 04:53:40,865 INFO [RS:1;jenkins-hbase4:37995] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-05-10 04:53:40,873 INFO [RS:0;jenkins-hbase4:36665] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-05-10 04:53:40,873 INFO [RS:2;jenkins-hbase4:37823] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-05-10 04:53:40,873 INFO [RS:1;jenkins-hbase4:37995] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-05-10 04:53:40,873 DEBUG [RS:0;jenkins-hbase4:36665] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-05-10 04:53:40,873 DEBUG [RS:2;jenkins-hbase4:37823] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-05-10 04:53:40,873 DEBUG [RS:0;jenkins-hbase4:36665] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-05-10 04:53:40,873 DEBUG [RS:1;jenkins-hbase4:37995] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-05-10 04:53:40,874 DEBUG [RS:0;jenkins-hbase4:36665] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-05-10 04:53:40,873 DEBUG [RS:2;jenkins-hbase4:37823] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-05-10 04:53:40,874 DEBUG [RS:1;jenkins-hbase4:37995] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-05-10 04:53:40,874 DEBUG [RS:0;jenkins-hbase4:36665] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-05-10 04:53:40,874 DEBUG [RS:1;jenkins-hbase4:37995] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-05-10 04:53:40,874 DEBUG [RS:0;jenkins-hbase4:36665] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-05-10 04:53:40,874 DEBUG [RS:1;jenkins-hbase4:37995] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-05-10 04:53:40,874 DEBUG [RS:0;jenkins-hbase4:36665] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-05-10 04:53:40,874 DEBUG [RS:1;jenkins-hbase4:37995] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-05-10 04:53:40,874 DEBUG [RS:2;jenkins-hbase4:37823] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-05-10 04:53:40,874 DEBUG [RS:1;jenkins-hbase4:37995] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-05-10 04:53:40,874 DEBUG [RS:0;jenkins-hbase4:36665] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-05-10 04:53:40,874 DEBUG [RS:1;jenkins-hbase4:37995] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-05-10 04:53:40,874 DEBUG [RS:2;jenkins-hbase4:37823] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-05-10 04:53:40,875 DEBUG [RS:1;jenkins-hbase4:37995] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-05-10 04:53:40,875 DEBUG [RS:0;jenkins-hbase4:36665] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-05-10 04:53:40,875 DEBUG [RS:1;jenkins-hbase4:37995] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-05-10 04:53:40,875 DEBUG [RS:0;jenkins-hbase4:36665] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-05-10 04:53:40,875 DEBUG [RS:1;jenkins-hbase4:37995] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-05-10 04:53:40,875 DEBUG [RS:0;jenkins-hbase4:36665] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-05-10 04:53:40,875 DEBUG [RS:2;jenkins-hbase4:37823] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-05-10 04:53:40,876 DEBUG [RS:2;jenkins-hbase4:37823] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase4:0, corePoolSize=2, maxPoolSize=2 2023-05-10 04:53:40,876 DEBUG [RS:2;jenkins-hbase4:37823] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-05-10 04:53:40,876 DEBUG [RS:2;jenkins-hbase4:37823] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-05-10 04:53:40,876 DEBUG [RS:2;jenkins-hbase4:37823] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-05-10 04:53:40,877 DEBUG [RS:2;jenkins-hbase4:37823] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase4:0, corePoolSize=1, maxPoolSize=1 2023-05-10 04:53:40,877 INFO [RS:0;jenkins-hbase4:36665] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-05-10 04:53:40,877 INFO [RS:0;jenkins-hbase4:36665] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-05-10 04:53:40,877 INFO [RS:0;jenkins-hbase4:36665] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-05-10 04:53:40,878 INFO [RS:1;jenkins-hbase4:37995] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-05-10 04:53:40,879 INFO [RS:1;jenkins-hbase4:37995] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-05-10 04:53:40,881 INFO [RS:1;jenkins-hbase4:37995] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-05-10 04:53:40,881 INFO [RS:2;jenkins-hbase4:37823] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-05-10 04:53:40,882 INFO [RS:2;jenkins-hbase4:37823] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-05-10 04:53:40,882 INFO [RS:2;jenkins-hbase4:37823] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-05-10 04:53:40,897 INFO [RS:2;jenkins-hbase4:37823] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-05-10 04:53:40,897 INFO [RS:0;jenkins-hbase4:36665] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-05-10 04:53:40,901 INFO [RS:1;jenkins-hbase4:37995] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-05-10 04:53:40,901 INFO [RS:2;jenkins-hbase4:37823] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,37823,1683694419534-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-05-10 04:53:40,901 INFO [RS:0;jenkins-hbase4:36665] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,36665,1683694419452-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-05-10 04:53:40,901 INFO [RS:1;jenkins-hbase4:37995] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,37995,1683694419499-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-05-10 04:53:40,919 INFO [RS:0;jenkins-hbase4:36665] regionserver.Replication(203): jenkins-hbase4.apache.org,36665,1683694419452 started 2023-05-10 04:53:40,920 INFO [RS:0;jenkins-hbase4:36665] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,36665,1683694419452, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:36665, sessionid=0x1001370791d0001 2023-05-10 04:53:40,920 DEBUG [RS:0;jenkins-hbase4:36665] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-05-10 04:53:40,920 DEBUG [RS:0;jenkins-hbase4:36665] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:40,920 DEBUG [RS:0;jenkins-hbase4:36665] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,36665,1683694419452' 2023-05-10 04:53:40,920 DEBUG [RS:0;jenkins-hbase4:36665] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:53:40,921 DEBUG [RS:0;jenkins-hbase4:36665] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:53:40,921 DEBUG [RS:0;jenkins-hbase4:36665] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-05-10 04:53:40,921 DEBUG [RS:0;jenkins-hbase4:36665] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-05-10 04:53:40,921 INFO [RS:2;jenkins-hbase4:37823] regionserver.Replication(203): jenkins-hbase4.apache.org,37823,1683694419534 started 2023-05-10 04:53:40,921 DEBUG [RS:0;jenkins-hbase4:36665] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:40,921 INFO [RS:2;jenkins-hbase4:37823] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,37823,1683694419534, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:37823, sessionid=0x1001370791d0003 2023-05-10 04:53:40,922 DEBUG [RS:0;jenkins-hbase4:36665] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,36665,1683694419452' 2023-05-10 04:53:40,922 DEBUG [RS:0;jenkins-hbase4:36665] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-05-10 04:53:40,922 DEBUG [RS:2;jenkins-hbase4:37823] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-05-10 04:53:40,922 DEBUG [RS:2;jenkins-hbase4:37823] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:40,922 DEBUG [RS:2;jenkins-hbase4:37823] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,37823,1683694419534' 2023-05-10 04:53:40,923 DEBUG [RS:2;jenkins-hbase4:37823] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:53:40,923 DEBUG [RS:0;jenkins-hbase4:36665] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-05-10 04:53:40,923 INFO [RS:1;jenkins-hbase4:37995] regionserver.Replication(203): jenkins-hbase4.apache.org,37995,1683694419499 started 2023-05-10 04:53:40,923 INFO [RS:1;jenkins-hbase4:37995] regionserver.HRegionServer(1637): Serving as jenkins-hbase4.apache.org,37995,1683694419499, RpcServer on jenkins-hbase4.apache.org/172.31.14.131:37995, sessionid=0x1001370791d0002 2023-05-10 04:53:40,923 DEBUG [RS:1;jenkins-hbase4:37995] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-05-10 04:53:40,924 DEBUG [RS:2;jenkins-hbase4:37823] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:53:40,924 DEBUG [RS:0;jenkins-hbase4:36665] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-05-10 04:53:40,924 DEBUG [RS:1;jenkins-hbase4:37995] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:40,924 DEBUG [RS:1;jenkins-hbase4:37995] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,37995,1683694419499' 2023-05-10 04:53:40,924 DEBUG [RS:1;jenkins-hbase4:37995] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:53:40,924 INFO [RS:0;jenkins-hbase4:36665] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-05-10 04:53:40,924 INFO [RS:0;jenkins-hbase4:36665] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-05-10 04:53:40,924 DEBUG [RS:2;jenkins-hbase4:37823] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-05-10 04:53:40,924 DEBUG [RS:2;jenkins-hbase4:37823] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-05-10 04:53:40,924 DEBUG [RS:2;jenkins-hbase4:37823] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:40,925 DEBUG [RS:1;jenkins-hbase4:37995] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:53:40,925 DEBUG [RS:2;jenkins-hbase4:37823] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,37823,1683694419534' 2023-05-10 04:53:40,925 DEBUG [RS:2;jenkins-hbase4:37823] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-05-10 04:53:40,925 DEBUG [RS:1;jenkins-hbase4:37995] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-05-10 04:53:40,926 DEBUG [RS:1;jenkins-hbase4:37995] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-05-10 04:53:40,926 DEBUG [RS:2;jenkins-hbase4:37823] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-05-10 04:53:40,926 DEBUG [RS:1;jenkins-hbase4:37995] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:40,926 DEBUG [RS:1;jenkins-hbase4:37995] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase4.apache.org,37995,1683694419499' 2023-05-10 04:53:40,926 DEBUG [RS:1;jenkins-hbase4:37995] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-05-10 04:53:40,926 DEBUG [RS:2;jenkins-hbase4:37823] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-05-10 04:53:40,926 INFO [RS:2;jenkins-hbase4:37823] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-05-10 04:53:40,927 INFO [RS:2;jenkins-hbase4:37823] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-05-10 04:53:40,928 DEBUG [RS:1;jenkins-hbase4:37995] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-05-10 04:53:40,928 DEBUG [RS:1;jenkins-hbase4:37995] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-05-10 04:53:40,928 INFO [RS:1;jenkins-hbase4:37995] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-05-10 04:53:40,928 INFO [RS:1;jenkins-hbase4:37995] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-05-10 04:53:41,004 DEBUG [jenkins-hbase4:37903] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=3, allServersCount=3 2023-05-10 04:53:41,007 DEBUG [jenkins-hbase4:37903] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-05-10 04:53:41,013 DEBUG [jenkins-hbase4:37903] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-05-10 04:53:41,013 DEBUG [jenkins-hbase4:37903] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-05-10 04:53:41,013 DEBUG [jenkins-hbase4:37903] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-05-10 04:53:41,013 DEBUG [jenkins-hbase4:37903] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-05-10 04:53:41,016 INFO [PEWorker-1] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,37995,1683694419499, state=OPENING 2023-05-10 04:53:41,023 DEBUG [PEWorker-1] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-05-10 04:53:41,031 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-05-10 04:53:41,031 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-05-10 04:53:41,035 INFO [RS:2;jenkins-hbase4:37823] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C37823%2C1683694419534, suffix=, logDir=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/WALs/jenkins-hbase4.apache.org,37823,1683694419534, archiveDir=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/oldWALs, maxLogs=32 2023-05-10 04:53:41,036 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,37995,1683694419499}] 2023-05-10 04:53:41,035 INFO [RS:0;jenkins-hbase4:36665] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C36665%2C1683694419452, suffix=, logDir=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/WALs/jenkins-hbase4.apache.org,36665,1683694419452, archiveDir=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/oldWALs, maxLogs=32 2023-05-10 04:53:41,037 INFO [RS:1;jenkins-hbase4:37995] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C37995%2C1683694419499, suffix=, logDir=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/WALs/jenkins-hbase4.apache.org,37995,1683694419499, archiveDir=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/oldWALs, maxLogs=32 2023-05-10 04:53:41,082 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:41547,DS-a5b310f7-364e-4271-a65f-70861aa844d7,DISK] 2023-05-10 04:53:41,082 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43517,DS-6547f095-4b0d-4558-8c35-c9b2bf842d27,DISK] 2023-05-10 04:53:41,082 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:42419,DS-a3c93b79-41c5-4541-ba4f-e18855846a32,DISK] 2023-05-10 04:53:41,084 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43517,DS-6547f095-4b0d-4558-8c35-c9b2bf842d27,DISK] 2023-05-10 04:53:41,088 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:42419,DS-a3c93b79-41c5-4541-ba4f-e18855846a32,DISK] 2023-05-10 04:53:41,088 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:41547,DS-a5b310f7-364e-4271-a65f-70861aa844d7,DISK] 2023-05-10 04:53:41,089 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43517,DS-6547f095-4b0d-4558-8c35-c9b2bf842d27,DISK] 2023-05-10 04:53:41,089 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:42419,DS-a3c93b79-41c5-4541-ba4f-e18855846a32,DISK] 2023-05-10 04:53:41,089 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:41547,DS-a5b310f7-364e-4271-a65f-70861aa844d7,DISK] 2023-05-10 04:53:41,101 INFO [RS:2;jenkins-hbase4:37823] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/WALs/jenkins-hbase4.apache.org,37823,1683694419534/jenkins-hbase4.apache.org%2C37823%2C1683694419534.1683694421041 2023-05-10 04:53:41,101 INFO [RS:1;jenkins-hbase4:37995] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/WALs/jenkins-hbase4.apache.org,37995,1683694419499/jenkins-hbase4.apache.org%2C37995%2C1683694419499.1683694421042 2023-05-10 04:53:41,103 DEBUG [RS:2;jenkins-hbase4:37823] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:41547,DS-a5b310f7-364e-4271-a65f-70861aa844d7,DISK], DatanodeInfoWithStorage[127.0.0.1:43517,DS-6547f095-4b0d-4558-8c35-c9b2bf842d27,DISK], DatanodeInfoWithStorage[127.0.0.1:42419,DS-a3c93b79-41c5-4541-ba4f-e18855846a32,DISK]] 2023-05-10 04:53:41,104 DEBUG [RS:1;jenkins-hbase4:37995] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:43517,DS-6547f095-4b0d-4558-8c35-c9b2bf842d27,DISK], DatanodeInfoWithStorage[127.0.0.1:42419,DS-a3c93b79-41c5-4541-ba4f-e18855846a32,DISK], DatanodeInfoWithStorage[127.0.0.1:41547,DS-a5b310f7-364e-4271-a65f-70861aa844d7,DISK]] 2023-05-10 04:53:41,104 INFO [RS:0;jenkins-hbase4:36665] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/WALs/jenkins-hbase4.apache.org,36665,1683694419452/jenkins-hbase4.apache.org%2C36665%2C1683694419452.1683694421041 2023-05-10 04:53:41,108 DEBUG [RS:0;jenkins-hbase4:36665] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:41547,DS-a5b310f7-364e-4271-a65f-70861aa844d7,DISK], DatanodeInfoWithStorage[127.0.0.1:42419,DS-a3c93b79-41c5-4541-ba4f-e18855846a32,DISK], DatanodeInfoWithStorage[127.0.0.1:43517,DS-6547f095-4b0d-4558-8c35-c9b2bf842d27,DISK]] 2023-05-10 04:53:41,223 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:41,227 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-05-10 04:53:41,232 INFO [RS-EventLoopGroup-4-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:38832, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-05-10 04:53:41,245 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-05-10 04:53:41,245 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-05-10 04:53:41,249 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase4.apache.org%2C37995%2C1683694419499.meta, suffix=.meta, logDir=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/WALs/jenkins-hbase4.apache.org,37995,1683694419499, archiveDir=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/oldWALs, maxLogs=32 2023-05-10 04:53:41,270 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:42419,DS-a3c93b79-41c5-4541-ba4f-e18855846a32,DISK] 2023-05-10 04:53:41,271 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:41547,DS-a5b310f7-364e-4271-a65f-70861aa844d7,DISK] 2023-05-10 04:53:41,272 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:43517,DS-6547f095-4b0d-4558-8c35-c9b2bf842d27,DISK] 2023-05-10 04:53:41,278 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/WALs/jenkins-hbase4.apache.org,37995,1683694419499/jenkins-hbase4.apache.org%2C37995%2C1683694419499.meta.1683694421250.meta 2023-05-10 04:53:41,279 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:42419,DS-a3c93b79-41c5-4541-ba4f-e18855846a32,DISK], DatanodeInfoWithStorage[127.0.0.1:41547,DS-a5b310f7-364e-4271-a65f-70861aa844d7,DISK], DatanodeInfoWithStorage[127.0.0.1:43517,DS-6547f095-4b0d-4558-8c35-c9b2bf842d27,DISK]] 2023-05-10 04:53:41,279 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-05-10 04:53:41,284 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-05-10 04:53:41,285 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-05-10 04:53:41,300 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-05-10 04:53:41,302 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-05-10 04:53:41,307 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-05-10 04:53:41,307 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-10 04:53:41,307 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-05-10 04:53:41,307 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-05-10 04:53:41,310 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-05-10 04:53:41,313 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/hbase/meta/1588230740/info 2023-05-10 04:53:41,313 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/hbase/meta/1588230740/info 2023-05-10 04:53:41,314 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-05-10 04:53:41,315 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-10 04:53:41,315 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-05-10 04:53:41,317 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/hbase/meta/1588230740/rep_barrier 2023-05-10 04:53:41,317 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/hbase/meta/1588230740/rep_barrier 2023-05-10 04:53:41,317 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-05-10 04:53:41,318 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-10 04:53:41,318 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-05-10 04:53:41,320 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/hbase/meta/1588230740/table 2023-05-10 04:53:41,320 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/hbase/meta/1588230740/table 2023-05-10 04:53:41,320 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-05-10 04:53:41,321 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-10 04:53:41,323 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/hbase/meta/1588230740 2023-05-10 04:53:41,325 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/hbase/meta/1588230740 2023-05-10 04:53:41,330 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-05-10 04:53:41,334 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-05-10 04:53:41,336 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73367717, jitterRate=0.09326417744159698}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-05-10 04:53:41,336 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-05-10 04:53:41,349 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2334): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1683694421215 2023-05-10 04:53:41,366 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2361): Finished post open deploy task for hbase:meta,,1.1588230740 2023-05-10 04:53:41,367 INFO [RS_OPEN_META-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-05-10 04:53:41,368 INFO [PEWorker-1] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase4.apache.org,37995,1683694419499, state=OPEN 2023-05-10 04:53:41,370 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-05-10 04:53:41,370 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-05-10 04:53:41,374 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=3, resume processing ppid=2 2023-05-10 04:53:41,375 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase4.apache.org,37995,1683694419499 in 335 msec 2023-05-10 04:53:41,381 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=2, resume processing ppid=1 2023-05-10 04:53:41,381 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 542 msec 2023-05-10 04:53:41,386 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 838 msec 2023-05-10 04:53:41,386 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1683694421386, completionTime=-1 2023-05-10 04:53:41,386 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=3; waited=0ms, expected min=3 server(s), max=3 server(s), master is running 2023-05-10 04:53:41,387 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1517): Joining cluster... 2023-05-10 04:53:41,453 DEBUG [hconnection-0x4a1c5cf4-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-05-10 04:53:41,456 INFO [RS-EventLoopGroup-4-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:38838, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-05-10 04:53:41,472 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-10 04:53:41,472 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683694481472 2023-05-10 04:53:41,472 INFO [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683694541472 2023-05-10 04:53:41,472 INFO [master/jenkins-hbase4:0:becomeActiveMaster] assignment.AssignmentManager(1536): Joined the cluster in 85 msec 2023-05-10 04:53:41,494 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,37903,1683694418326-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-05-10 04:53:41,495 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,37903,1683694418326-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-05-10 04:53:41,495 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,37903,1683694418326-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-05-10 04:53:41,496 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase4:37903, period=300000, unit=MILLISECONDS is enabled. 2023-05-10 04:53:41,497 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-05-10 04:53:41,507 DEBUG [master/jenkins-hbase4:0.Chore.1] janitor.CatalogJanitor(175): 2023-05-10 04:53:41,515 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-05-10 04:53:41,517 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(2148): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-05-10 04:53:41,526 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-05-10 04:53:41,528 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-05-10 04:53:41,531 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-05-10 04:53:41,555 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/hbase/namespace/ccfda81dc2f13ffc1429bef67d349028 2023-05-10 04:53:41,558 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/hbase/namespace/ccfda81dc2f13ffc1429bef67d349028 empty. 2023-05-10 04:53:41,558 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/hbase/namespace/ccfda81dc2f13ffc1429bef67d349028 2023-05-10 04:53:41,559 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-05-10 04:53:41,592 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-05-10 04:53:41,594 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7675): creating {ENCODED => ccfda81dc2f13ffc1429bef67d349028, NAME => 'hbase:namespace,,1683694421516.ccfda81dc2f13ffc1429bef67d349028.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp 2023-05-10 04:53:41,613 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1683694421516.ccfda81dc2f13ffc1429bef67d349028.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-10 04:53:41,613 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1604): Closing ccfda81dc2f13ffc1429bef67d349028, disabling compactions & flushes 2023-05-10 04:53:41,613 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1683694421516.ccfda81dc2f13ffc1429bef67d349028. 2023-05-10 04:53:41,613 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1683694421516.ccfda81dc2f13ffc1429bef67d349028. 2023-05-10 04:53:41,613 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1683694421516.ccfda81dc2f13ffc1429bef67d349028. after waiting 0 ms 2023-05-10 04:53:41,613 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1683694421516.ccfda81dc2f13ffc1429bef67d349028. 2023-05-10 04:53:41,613 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1838): Closed hbase:namespace,,1683694421516.ccfda81dc2f13ffc1429bef67d349028. 2023-05-10 04:53:41,614 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1558): Region close journal for ccfda81dc2f13ffc1429bef67d349028: 2023-05-10 04:53:41,618 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-05-10 04:53:41,633 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1683694421516.ccfda81dc2f13ffc1429bef67d349028.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1683694421621"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683694421621"}]},"ts":"1683694421621"} 2023-05-10 04:53:41,662 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-05-10 04:53:41,664 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-05-10 04:53:41,668 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683694421664"}]},"ts":"1683694421664"} 2023-05-10 04:53:41,672 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-05-10 04:53:41,677 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-05-10 04:53:41,678 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-05-10 04:53:41,678 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-05-10 04:53:41,678 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-05-10 04:53:41,678 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-05-10 04:53:41,681 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=ccfda81dc2f13ffc1429bef67d349028, ASSIGN}] 2023-05-10 04:53:41,684 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=ccfda81dc2f13ffc1429bef67d349028, ASSIGN 2023-05-10 04:53:41,685 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=ccfda81dc2f13ffc1429bef67d349028, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,36665,1683694419452; forceNewPlan=false, retain=false 2023-05-10 04:53:41,838 INFO [jenkins-hbase4:37903] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-05-10 04:53:41,839 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=ccfda81dc2f13ffc1429bef67d349028, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:41,839 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1683694421516.ccfda81dc2f13ffc1429bef67d349028.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1683694421839"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683694421839"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683694421839"}]},"ts":"1683694421839"} 2023-05-10 04:53:41,844 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure ccfda81dc2f13ffc1429bef67d349028, server=jenkins-hbase4.apache.org,36665,1683694419452}] 2023-05-10 04:53:42,000 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:42,000 DEBUG [RSProcedureDispatcher-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-05-10 04:53:42,004 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:42514, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-05-10 04:53:42,009 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1683694421516.ccfda81dc2f13ffc1429bef67d349028. 2023-05-10 04:53:42,010 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => ccfda81dc2f13ffc1429bef67d349028, NAME => 'hbase:namespace,,1683694421516.ccfda81dc2f13ffc1429bef67d349028.', STARTKEY => '', ENDKEY => ''} 2023-05-10 04:53:42,012 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-05-10 04:53:42,013 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace ccfda81dc2f13ffc1429bef67d349028 2023-05-10 04:53:42,013 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1683694421516.ccfda81dc2f13ffc1429bef67d349028.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-10 04:53:42,013 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for ccfda81dc2f13ffc1429bef67d349028 2023-05-10 04:53:42,013 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for ccfda81dc2f13ffc1429bef67d349028 2023-05-10 04:53:42,015 INFO [StoreOpener-ccfda81dc2f13ffc1429bef67d349028-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region ccfda81dc2f13ffc1429bef67d349028 2023-05-10 04:53:42,018 DEBUG [StoreOpener-ccfda81dc2f13ffc1429bef67d349028-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/hbase/namespace/ccfda81dc2f13ffc1429bef67d349028/info 2023-05-10 04:53:42,018 DEBUG [StoreOpener-ccfda81dc2f13ffc1429bef67d349028-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/hbase/namespace/ccfda81dc2f13ffc1429bef67d349028/info 2023-05-10 04:53:42,018 INFO [StoreOpener-ccfda81dc2f13ffc1429bef67d349028-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ccfda81dc2f13ffc1429bef67d349028 columnFamilyName info 2023-05-10 04:53:42,019 INFO [StoreOpener-ccfda81dc2f13ffc1429bef67d349028-1] regionserver.HStore(310): Store=ccfda81dc2f13ffc1429bef67d349028/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-10 04:53:42,021 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/hbase/namespace/ccfda81dc2f13ffc1429bef67d349028 2023-05-10 04:53:42,022 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/hbase/namespace/ccfda81dc2f13ffc1429bef67d349028 2023-05-10 04:53:42,026 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for ccfda81dc2f13ffc1429bef67d349028 2023-05-10 04:53:42,030 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/hbase/namespace/ccfda81dc2f13ffc1429bef67d349028/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-05-10 04:53:42,031 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened ccfda81dc2f13ffc1429bef67d349028; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64294927, jitterRate=-0.04193092882633209}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-05-10 04:53:42,031 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for ccfda81dc2f13ffc1429bef67d349028: 2023-05-10 04:53:42,033 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2334): Post open deploy tasks for hbase:namespace,,1683694421516.ccfda81dc2f13ffc1429bef67d349028., pid=6, masterSystemTime=1683694421999 2023-05-10 04:53:42,038 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2361): Finished post open deploy task for hbase:namespace,,1683694421516.ccfda81dc2f13ffc1429bef67d349028. 2023-05-10 04:53:42,038 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1683694421516.ccfda81dc2f13ffc1429bef67d349028. 2023-05-10 04:53:42,040 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=ccfda81dc2f13ffc1429bef67d349028, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:42,040 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1683694421516.ccfda81dc2f13ffc1429bef67d349028.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1683694422039"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1683694422039"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1683694422039"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683694422039"}]},"ts":"1683694422039"} 2023-05-10 04:53:42,048 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=6, resume processing ppid=5 2023-05-10 04:53:42,048 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure ccfda81dc2f13ffc1429bef67d349028, server=jenkins-hbase4.apache.org,36665,1683694419452 in 200 msec 2023-05-10 04:53:42,052 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=5, resume processing ppid=4 2023-05-10 04:53:42,052 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=ccfda81dc2f13ffc1429bef67d349028, ASSIGN in 367 msec 2023-05-10 04:53:42,054 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-05-10 04:53:42,054 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683694422054"}]},"ts":"1683694422054"} 2023-05-10 04:53:42,056 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-05-10 04:53:42,059 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-05-10 04:53:42,062 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 541 msec 2023-05-10 04:53:42,129 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/namespace 2023-05-10 04:53:42,131 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/namespace 2023-05-10 04:53:42,131 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-05-10 04:53:42,153 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-05-10 04:53:42,156 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:42520, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-05-10 04:53:42,172 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-05-10 04:53:42,189 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-05-10 04:53:42,197 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 34 msec 2023-05-10 04:53:42,206 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-05-10 04:53:42,217 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-05-10 04:53:42,221 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 15 msec 2023-05-10 04:53:42,233 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-05-10 04:53:42,235 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2023-05-10 04:53:42,235 INFO [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 2.634sec 2023-05-10 04:53:42,238 INFO [master/jenkins-hbase4:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-05-10 04:53:42,239 INFO [master/jenkins-hbase4:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-05-10 04:53:42,239 INFO [master/jenkins-hbase4:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-05-10 04:53:42,241 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,37903,1683694418326-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-05-10 04:53:42,241 INFO [master/jenkins-hbase4:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase4.apache.org,37903,1683694418326-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-05-10 04:53:42,249 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-05-10 04:53:42,270 DEBUG [Listener at localhost/45007] zookeeper.ReadOnlyZKClient(139): Connect 0x0bbebf78 to 127.0.0.1:51177 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-05-10 04:53:42,275 DEBUG [Listener at localhost/45007] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@5ea55edf, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-05-10 04:53:42,289 DEBUG [hconnection-0x248fdd40-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-05-10 04:53:42,302 INFO [RS-EventLoopGroup-4-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:38848, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-05-10 04:53:42,311 INFO [Listener at localhost/45007] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase4.apache.org,37903,1683694418326 2023-05-10 04:53:42,342 INFO [Listener at localhost/45007] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionUnevenDaughter Thread=360, OpenFileDescriptor=695, MaxFileDescriptor=60000, SystemLoadAverage=344, ProcessCount=175, AvailableMemoryMB=9782 2023-05-10 04:53:42,347 WARN [Listener at localhost/45007] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-05-10 04:53:42,355 DEBUG [Listener at localhost/45007] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-05-10 04:53:42,359 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:39844, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-05-10 04:53:42,375 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/balancer 2023-05-10 04:53:42,375 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-05-10 04:53:42,376 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-05-10 04:53:42,382 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(1029): Stored pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:53:42,384 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_PRE_OPERATION 2023-05-10 04:53:42,386 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-05-10 04:53:42,389 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522 2023-05-10 04:53:42,390 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522 empty. 2023-05-10 04:53:42,392 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522 2023-05-10 04:53:42,392 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionUnevenDaughter regions 2023-05-10 04:53:42,414 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionUnevenDaughter/.tabledesc/.tableinfo.0000000001 2023-05-10 04:53:42,416 INFO [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(7675): creating {ENCODED => 93c2cc329b1fe1893cfbc28c56960522, NAME => 'testSplitTableRegionUnevenDaughter,,1683694422380.93c2cc329b1fe1893cfbc28c56960522.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionUnevenDaughter', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp 2023-05-10 04:53:42,441 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(866): Instantiated testSplitTableRegionUnevenDaughter,,1683694422380.93c2cc329b1fe1893cfbc28c56960522.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-10 04:53:42,441 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1604): Closing 93c2cc329b1fe1893cfbc28c56960522, disabling compactions & flushes 2023-05-10 04:53:42,441 INFO [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1626): Closing region testSplitTableRegionUnevenDaughter,,1683694422380.93c2cc329b1fe1893cfbc28c56960522. 2023-05-10 04:53:42,441 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,,1683694422380.93c2cc329b1fe1893cfbc28c56960522. 2023-05-10 04:53:42,441 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionUnevenDaughter,,1683694422380.93c2cc329b1fe1893cfbc28c56960522. after waiting 0 ms 2023-05-10 04:53:42,441 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionUnevenDaughter,,1683694422380.93c2cc329b1fe1893cfbc28c56960522. 2023-05-10 04:53:42,441 INFO [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1838): Closed testSplitTableRegionUnevenDaughter,,1683694422380.93c2cc329b1fe1893cfbc28c56960522. 2023-05-10 04:53:42,441 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1558): Region close journal for 93c2cc329b1fe1893cfbc28c56960522: 2023-05-10 04:53:42,446 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_ADD_TO_META 2023-05-10 04:53:42,448 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,,1683694422380.93c2cc329b1fe1893cfbc28c56960522.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1683694422448"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683694422448"}]},"ts":"1683694422448"} 2023-05-10 04:53:42,451 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-05-10 04:53:42,452 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-05-10 04:53:42,453 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683694422453"}]},"ts":"1683694422453"} 2023-05-10 04:53:42,455 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=ENABLING in hbase:meta 2023-05-10 04:53:42,459 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-05-10 04:53:42,461 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-05-10 04:53:42,461 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-05-10 04:53:42,461 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-05-10 04:53:42,461 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-05-10 04:53:42,462 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=93c2cc329b1fe1893cfbc28c56960522, ASSIGN}] 2023-05-10 04:53:42,465 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=93c2cc329b1fe1893cfbc28c56960522, ASSIGN 2023-05-10 04:53:42,466 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=93c2cc329b1fe1893cfbc28c56960522, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,37823,1683694419534; forceNewPlan=false, retain=false 2023-05-10 04:53:42,616 INFO [jenkins-hbase4:37903] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-05-10 04:53:42,617 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=93c2cc329b1fe1893cfbc28c56960522, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:42,618 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1683694422380.93c2cc329b1fe1893cfbc28c56960522.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1683694422617"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683694422617"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683694422617"}]},"ts":"1683694422617"} 2023-05-10 04:53:42,621 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=11, ppid=10, state=RUNNABLE; OpenRegionProcedure 93c2cc329b1fe1893cfbc28c56960522, server=jenkins-hbase4.apache.org,37823,1683694419534}] 2023-05-10 04:53:42,774 DEBUG [RSProcedureDispatcher-pool-2] master.ServerManager(712): New admin connection to jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:42,775 DEBUG [RSProcedureDispatcher-pool-2] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-05-10 04:53:42,776 INFO [RS-EventLoopGroup-5-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:55402, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-05-10 04:53:42,783 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionUnevenDaughter,,1683694422380.93c2cc329b1fe1893cfbc28c56960522. 2023-05-10 04:53:42,783 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 93c2cc329b1fe1893cfbc28c56960522, NAME => 'testSplitTableRegionUnevenDaughter,,1683694422380.93c2cc329b1fe1893cfbc28c56960522.', STARTKEY => '', ENDKEY => ''} 2023-05-10 04:53:42,783 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-05-10 04:53:42,783 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionUnevenDaughter 93c2cc329b1fe1893cfbc28c56960522 2023-05-10 04:53:42,783 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionUnevenDaughter,,1683694422380.93c2cc329b1fe1893cfbc28c56960522.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-10 04:53:42,783 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 93c2cc329b1fe1893cfbc28c56960522 2023-05-10 04:53:42,784 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 93c2cc329b1fe1893cfbc28c56960522 2023-05-10 04:53:42,785 INFO [StoreOpener-93c2cc329b1fe1893cfbc28c56960522-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 93c2cc329b1fe1893cfbc28c56960522 2023-05-10 04:53:42,788 DEBUG [StoreOpener-93c2cc329b1fe1893cfbc28c56960522-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf1 2023-05-10 04:53:42,788 DEBUG [StoreOpener-93c2cc329b1fe1893cfbc28c56960522-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf1 2023-05-10 04:53:42,788 INFO [StoreOpener-93c2cc329b1fe1893cfbc28c56960522-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 93c2cc329b1fe1893cfbc28c56960522 columnFamilyName cf1 2023-05-10 04:53:42,789 INFO [StoreOpener-93c2cc329b1fe1893cfbc28c56960522-1] regionserver.HStore(310): Store=93c2cc329b1fe1893cfbc28c56960522/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-10 04:53:42,789 INFO [StoreOpener-93c2cc329b1fe1893cfbc28c56960522-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 93c2cc329b1fe1893cfbc28c56960522 2023-05-10 04:53:42,791 DEBUG [StoreOpener-93c2cc329b1fe1893cfbc28c56960522-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf2 2023-05-10 04:53:42,791 DEBUG [StoreOpener-93c2cc329b1fe1893cfbc28c56960522-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf2 2023-05-10 04:53:42,792 INFO [StoreOpener-93c2cc329b1fe1893cfbc28c56960522-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 93c2cc329b1fe1893cfbc28c56960522 columnFamilyName cf2 2023-05-10 04:53:42,792 INFO [StoreOpener-93c2cc329b1fe1893cfbc28c56960522-1] regionserver.HStore(310): Store=93c2cc329b1fe1893cfbc28c56960522/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-10 04:53:42,795 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522 2023-05-10 04:53:42,796 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522 2023-05-10 04:53:42,799 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionUnevenDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-05-10 04:53:42,801 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 93c2cc329b1fe1893cfbc28c56960522 2023-05-10 04:53:42,804 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-05-10 04:53:42,805 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 93c2cc329b1fe1893cfbc28c56960522; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=75304784, jitterRate=0.12212872505187988}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-05-10 04:53:42,805 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 93c2cc329b1fe1893cfbc28c56960522: 2023-05-10 04:53:42,807 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2334): Post open deploy tasks for testSplitTableRegionUnevenDaughter,,1683694422380.93c2cc329b1fe1893cfbc28c56960522., pid=11, masterSystemTime=1683694422774 2023-05-10 04:53:42,811 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2361): Finished post open deploy task for testSplitTableRegionUnevenDaughter,,1683694422380.93c2cc329b1fe1893cfbc28c56960522. 2023-05-10 04:53:42,812 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionUnevenDaughter,,1683694422380.93c2cc329b1fe1893cfbc28c56960522. 2023-05-10 04:53:42,813 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=93c2cc329b1fe1893cfbc28c56960522, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:42,814 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionUnevenDaughter,,1683694422380.93c2cc329b1fe1893cfbc28c56960522.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1683694422813"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1683694422813"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1683694422813"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683694422813"}]},"ts":"1683694422813"} 2023-05-10 04:53:42,820 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=11, resume processing ppid=10 2023-05-10 04:53:42,820 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=11, ppid=10, state=SUCCESS; OpenRegionProcedure 93c2cc329b1fe1893cfbc28c56960522, server=jenkins-hbase4.apache.org,37823,1683694419534 in 195 msec 2023-05-10 04:53:42,823 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=10, resume processing ppid=9 2023-05-10 04:53:42,823 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=10, ppid=9, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=93c2cc329b1fe1893cfbc28c56960522, ASSIGN in 359 msec 2023-05-10 04:53:42,824 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-05-10 04:53:42,825 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683694422824"}]},"ts":"1683694422824"} 2023-05-10 04:53:42,827 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=ENABLED in hbase:meta 2023-05-10 04:53:42,831 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_POST_OPERATION 2023-05-10 04:53:42,833 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter in 451 msec 2023-05-10 04:53:42,887 DEBUG [Listener at localhost/45007] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-05-10 04:53:42,892 INFO [RS-EventLoopGroup-5-3] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:55416, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-05-10 04:53:42,912 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-10 04:53:42,919 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-05-10 04:53:42,920 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-05-10 04:53:42,920 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:53:42,920 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-05-10 04:53:42,920 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-05-10 04:53:42,921 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:42,921 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:42,922 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:53:42,922 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:53:42,922 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:42,922 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:53:42,922 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:53:42,923 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:53:42,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:53:42,922 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:53:42,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:53:42,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:53:42,923 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:42,923 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-10 04:53:42,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:42,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:42,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:42,924 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:42,924 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:42,924 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:42,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:53:42,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:53:42,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:53:42,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:42,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:42,925 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-05-10 04:53:42,925 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-05-10 04:53:42,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:42,925 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-05-10 04:53:42,926 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-05-10 04:53:42,926 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-05-10 04:53:42,927 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-05-10 04:53:42,927 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-05-10 04:53:42,927 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:53:42,927 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-05-10 04:53:42,927 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-05-10 04:53:42,927 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:53:42,927 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-05-10 04:53:42,928 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-05-10 04:53:42,928 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-05-10 04:53:42,928 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:53:42,928 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37995,1683694419499' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-05-10 04:53:42,928 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-05-10 04:53:42,929 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-05-10 04:53:42,929 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36665,1683694419452' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-05-10 04:53:42,930 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-10 04:53:42,930 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-10 04:53:42,930 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1683694422380.93c2cc329b1fe1893cfbc28c56960522. 2023-05-10 04:53:42,930 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1683694422380.93c2cc329b1fe1893cfbc28c56960522. started... 2023-05-10 04:53:42,931 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:42,932 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 93c2cc329b1fe1893cfbc28c56960522 2/2 column families, dataSize=120 B heapSize=928 B 2023-05-10 04:53:42,932 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:42,932 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:42,932 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:53:42,932 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:42,932 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:53:43,026 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/.tmp/cf1/9333fac35a6e43f2ae1eb05c2a8c43e2 2023-05-10 04:53:43,491 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/.tmp/cf2/b1cbcf3117244e049435fa8e701a7a80 2023-05-10 04:53:43,500 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/.tmp/cf1/9333fac35a6e43f2ae1eb05c2a8c43e2 as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf1/9333fac35a6e43f2ae1eb05c2a8c43e2 2023-05-10 04:53:43,510 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf1/9333fac35a6e43f2ae1eb05c2a8c43e2, entries=2, sequenceid=6, filesize=4.8 K 2023-05-10 04:53:43,513 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/.tmp/cf2/b1cbcf3117244e049435fa8e701a7a80 as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf2/b1cbcf3117244e049435fa8e701a7a80 2023-05-10 04:53:43,521 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf2/b1cbcf3117244e049435fa8e701a7a80, entries=2, sequenceid=6, filesize=4.8 K 2023-05-10 04:53:43,523 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 93c2cc329b1fe1893cfbc28c56960522 in 591ms, sequenceid=6, compaction requested=false 2023-05-10 04:53:43,524 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegionUnevenDaughter' 2023-05-10 04:53:43,526 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 93c2cc329b1fe1893cfbc28c56960522: 2023-05-10 04:53:43,526 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1683694422380.93c2cc329b1fe1893cfbc28c56960522. 2023-05-10 04:53:43,526 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-10 04:53:43,527 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-10 04:53:43,527 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:43,527 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-05-10 04:53:43,527 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37823,1683694419534' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-05-10 04:53:43,530 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,530 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:43,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:43,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:53:43,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:53:43,531 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,531 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:53:43,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:53:43,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:53:43,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:43,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:43,533 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:43,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:53:43,534 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37823,1683694419534' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-05-10 04:53:43,534 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-05-10 04:53:43,534 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5d3f8225[Count = 0] remaining members to acquire global barrier 2023-05-10 04:53:43,534 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,539 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,539 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,539 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,539 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,539 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,540 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-05-10 04:53:43,540 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:43,540 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,540 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-10 04:53:43,540 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-05-10 04:53:43,540 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-05-10 04:53:43,540 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-05-10 04:53:43,540 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,37823,1683694419534' in zk 2023-05-10 04:53:43,540 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,540 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,37995,1683694419499' in zk 2023-05-10 04:53:43,540 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-05-10 04:53:43,540 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-05-10 04:53:43,540 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,36665,1683694419452' in zk 2023-05-10 04:53:43,544 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-05-10 04:53:43,544 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:43,544 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:53:43,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:43,545 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-05-10 04:53:43,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:53:43,545 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:53:43,545 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-05-10 04:53:43,545 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-05-10 04:53:43,545 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:53:43,545 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:53:43,546 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-05-10 04:53:43,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:53:43,546 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-05-10 04:53:43,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:53:43,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:43,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:43,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:43,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:53:43,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:43,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:43,550 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:43,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,37823,1683694419534': 2023-05-10 04:53:43,551 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37823,1683694419534' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-05-10 04:53:43,551 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-05-10 04:53:43,551 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-10 04:53:43,551 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-10 04:53:43,552 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,552 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-10 04:53:43,556 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,556 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,556 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,556 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,556 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:53:43,556 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:53:43,556 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,556 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,556 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:53:43,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:53:43,556 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:53:43,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,556 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:53:43,557 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:53:43,557 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:43,557 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:53:43,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:53:43,557 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:53:43,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:53:43,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:53:43,557 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:43,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,558 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:43,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:53:43,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:43,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:43,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:43,561 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:43,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:53:43,561 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:43,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,562 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:43,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:43,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:43,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:43,576 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:53:43,576 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:53:43,576 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:53:43,576 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:53:43,576 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:53:43,576 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:43,576 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,577 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:43,577 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:43,577 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,577 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:53:43,576 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:53:43,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:53:43,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:53:43,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:53:43,577 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:53:43,578 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:43,578 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,578 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:53:43,578 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:43,578 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:43,578 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,578 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,578 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:53:43,578 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:53:43,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:53:43,579 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:53:43,578 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:53:43,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:53:43,579 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-05-10 04:53:43,579 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-10 04:53:43,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:53:43,581 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-05-10 04:53:43,581 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-10 04:53:43,682 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-10 04:53:43,686 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-10 04:53:43,711 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-10 04:53:43,714 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-05-10 04:53:43,714 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,714 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-05-10 04:53:43,714 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:53:43,715 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-05-10 04:53:43,715 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-05-10 04:53:43,715 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,715 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,717 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:53:43,717 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:53:43,717 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:53:43,717 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:43,717 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:53:43,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:53:43,717 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:53:43,717 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:53:43,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:53:43,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:53:43,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,718 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:43,718 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-10 04:53:43,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,718 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,719 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,719 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:53:43,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,719 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:53:43,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:53:43,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,719 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,719 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,719 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-05-10 04:53:43,720 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,720 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-05-10 04:53:43,720 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:53:43,721 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-05-10 04:53:43,721 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-05-10 04:53:43,721 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-05-10 04:53:43,721 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-05-10 04:53:43,721 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,721 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36665,1683694419452' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-05-10 04:53:43,721 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-05-10 04:53:43,721 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,721 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:53:43,721 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-05-10 04:53:43,722 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-05-10 04:53:43,722 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:53:43,722 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-10 04:53:43,723 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1683694422380.93c2cc329b1fe1893cfbc28c56960522. 2023-05-10 04:53:43,723 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-10 04:53:43,723 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-05-10 04:53:43,723 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1683694422380.93c2cc329b1fe1893cfbc28c56960522. started... 2023-05-10 04:53:43,724 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-05-10 04:53:43,724 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37995,1683694419499' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-05-10 04:53:43,724 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 93c2cc329b1fe1893cfbc28c56960522 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-10 04:53:43,725 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,725 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,726 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:53:43,727 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,727 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,727 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:53:43,743 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/.tmp/cf1/6fc2425571254b1b8437fabc5c780c71 2023-05-10 04:53:43,773 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/.tmp/cf2/bd68bbeaf0094fd0a415560924da2bce 2023-05-10 04:53:43,782 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/.tmp/cf1/6fc2425571254b1b8437fabc5c780c71 as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf1/6fc2425571254b1b8437fabc5c780c71 2023-05-10 04:53:43,790 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf1/6fc2425571254b1b8437fabc5c780c71, entries=10, sequenceid=19, filesize=5.0 K 2023-05-10 04:53:43,792 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/.tmp/cf2/bd68bbeaf0094fd0a415560924da2bce as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf2/bd68bbeaf0094fd0a415560924da2bce 2023-05-10 04:53:43,800 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf2/bd68bbeaf0094fd0a415560924da2bce, entries=10, sequenceid=19, filesize=5.0 K 2023-05-10 04:53:43,801 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 93c2cc329b1fe1893cfbc28c56960522 in 77ms, sequenceid=19, compaction requested=false 2023-05-10 04:53:43,801 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 93c2cc329b1fe1893cfbc28c56960522: 2023-05-10 04:53:43,801 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1683694422380.93c2cc329b1fe1893cfbc28c56960522. 2023-05-10 04:53:43,801 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-10 04:53:43,802 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-10 04:53:43,802 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:43,802 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-05-10 04:53:43,802 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37823,1683694419534' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-05-10 04:53:43,805 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,805 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:43,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:43,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:53:43,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:53:43,805 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,806 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:53:43,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:53:43,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:53:43,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:43,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:43,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:43,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:53:43,808 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37823,1683694419534' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-05-10 04:53:43,808 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@74afa0a5[Count = 0] remaining members to acquire global barrier 2023-05-10 04:53:43,808 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-05-10 04:53:43,808 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,809 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,809 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,809 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,809 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,809 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,810 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,810 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:43,810 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-10 04:53:43,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,810 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-05-10 04:53:43,810 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-05-10 04:53:43,810 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-05-10 04:53:43,810 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-05-10 04:53:43,810 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-05-10 04:53:43,810 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,37823,1683694419534' in zk 2023-05-10 04:53:43,810 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-05-10 04:53:43,810 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,36665,1683694419452' in zk 2023-05-10 04:53:43,810 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,37995,1683694419499' in zk 2023-05-10 04:53:43,812 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-05-10 04:53:43,812 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:43,812 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:53:43,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:43,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:53:43,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:53:43,813 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-05-10 04:53:43,813 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-05-10 04:53:43,813 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-05-10 04:53:43,813 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:53:43,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:53:43,814 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:53:43,814 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-05-10 04:53:43,814 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-05-10 04:53:43,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:53:43,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:43,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:43,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:43,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:53:43,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:43,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:43,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:43,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,37823,1683694419534': 2023-05-10 04:53:43,818 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37823,1683694419534' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-05-10 04:53:43,818 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-05-10 04:53:43,819 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-10 04:53:43,819 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-10 04:53:43,819 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,819 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-10 04:53:43,821 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,821 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,821 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,821 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:53:43,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,822 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,822 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,822 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,822 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:53:43,822 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,822 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:53:43,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:53:43,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:53:43,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,822 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:43,822 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:53:43,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:53:43,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:53:43,823 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:53:43,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:53:43,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,825 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:53:43,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:53:43,826 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:43,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,826 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:43,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:53:43,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:43,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:43,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:43,829 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:43,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:53:43,830 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:43,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,831 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:43,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:43,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:43,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:43,842 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:53:43,842 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:53:43,842 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:53:43,842 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:53:43,842 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:53:43,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:53:43,842 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:53:43,842 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:43,842 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:53:43,842 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:53:43,842 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:53:43,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:53:43,843 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:53:43,843 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:53:43,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:53:43,842 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:53:43,844 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:43,844 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:43,844 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,844 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:43,844 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,844 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:43,844 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:43,844 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,844 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:53:43,843 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-05-10 04:53:43,844 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-10 04:53:43,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:53:43,844 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,844 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:53:43,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:53:43,845 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-05-10 04:53:43,845 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-10 04:53:43,945 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-10 04:53:43,947 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-10 04:53:43,969 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-10 04:53:43,971 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-05-10 04:53:43,971 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,971 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-05-10 04:53:43,971 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:53:43,972 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-05-10 04:53:43,972 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-05-10 04:53:43,973 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,973 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,974 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:53:43,974 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:53:43,974 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:53:43,974 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:53:43,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:53:43,974 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:43,974 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:53:43,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:53:43,975 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:53:43,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:53:43,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,975 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:43,975 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-10 04:53:43,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,975 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,976 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,976 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,976 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:53:43,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:53:43,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:53:43,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,976 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,976 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,976 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,976 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,977 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-05-10 04:53:43,977 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,977 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-05-10 04:53:43,977 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-05-10 04:53:43,977 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:53:43,977 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,977 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-05-10 04:53:43,977 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:53:43,978 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-05-10 04:53:43,980 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,980 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-05-10 04:53:43,981 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-05-10 04:53:43,981 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:53:43,980 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-05-10 04:53:43,982 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-10 04:53:43,982 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-10 04:53:43,982 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-05-10 04:53:43,982 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-05-10 04:53:43,982 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1683694422380.93c2cc329b1fe1893cfbc28c56960522. 2023-05-10 04:53:43,982 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-05-10 04:53:43,982 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1683694422380.93c2cc329b1fe1893cfbc28c56960522. started... 2023-05-10 04:53:43,982 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37995,1683694419499' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-05-10 04:53:43,982 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36665,1683694419452' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-05-10 04:53:43,982 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 93c2cc329b1fe1893cfbc28c56960522 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-10 04:53:43,986 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,986 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,986 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,986 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:53:43,986 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:43,986 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:53:44,005 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/.tmp/cf1/be24ca36992745b7a1a3b25d9bc68d53 2023-05-10 04:53:44,032 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/.tmp/cf2/76f6f9416941403596b6ff3caccb861c 2023-05-10 04:53:44,041 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/.tmp/cf1/be24ca36992745b7a1a3b25d9bc68d53 as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf1/be24ca36992745b7a1a3b25d9bc68d53 2023-05-10 04:53:44,048 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf1/be24ca36992745b7a1a3b25d9bc68d53, entries=10, sequenceid=32, filesize=5.0 K 2023-05-10 04:53:44,049 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/.tmp/cf2/76f6f9416941403596b6ff3caccb861c as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf2/76f6f9416941403596b6ff3caccb861c 2023-05-10 04:53:44,058 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf2/76f6f9416941403596b6ff3caccb861c, entries=10, sequenceid=32, filesize=5.0 K 2023-05-10 04:53:44,059 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 93c2cc329b1fe1893cfbc28c56960522 in 77ms, sequenceid=32, compaction requested=true 2023-05-10 04:53:44,059 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 93c2cc329b1fe1893cfbc28c56960522: 2023-05-10 04:53:44,059 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1683694422380.93c2cc329b1fe1893cfbc28c56960522. 2023-05-10 04:53:44,059 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-10 04:53:44,060 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-10 04:53:44,060 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:44,060 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-05-10 04:53:44,060 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37823,1683694419534' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-05-10 04:53:44,062 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:44,062 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:44,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:53:44,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:53:44,062 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,063 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:53:44,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:53:44,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:53:44,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:44,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:44,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:44,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:53:44,065 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37823,1683694419534' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-05-10 04:53:44,065 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2f1d01c3[Count = 0] remaining members to acquire global barrier 2023-05-10 04:53:44,065 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-05-10 04:53:44,065 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,066 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,066 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,066 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,066 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,066 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,066 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,066 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-05-10 04:53:44,067 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-05-10 04:53:44,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,067 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,37823,1683694419534' in zk 2023-05-10 04:53:44,067 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-05-10 04:53:44,067 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,067 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-05-10 04:53:44,067 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-05-10 04:53:44,067 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-05-10 04:53:44,067 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:44,067 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,36665,1683694419452' in zk 2023-05-10 04:53:44,067 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,37995,1683694419499' in zk 2023-05-10 04:53:44,067 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-10 04:53:44,070 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:44,070 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-05-10 04:53:44,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:44,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:53:44,070 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:53:44,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:53:44,070 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-05-10 04:53:44,072 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-05-10 04:53:44,072 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:53:44,072 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-05-10 04:53:44,074 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-05-10 04:53:44,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:53:44,074 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:53:44,074 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-05-10 04:53:44,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:53:44,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:44,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:44,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:44,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:53:44,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:44,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:44,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:44,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,37823,1683694419534': 2023-05-10 04:53:44,078 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37823,1683694419534' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-05-10 04:53:44,078 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-05-10 04:53:44,078 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-10 04:53:44,078 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-10 04:53:44,078 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,078 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-10 04:53:44,080 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,080 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,080 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,080 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,080 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,080 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:53:44,080 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:53:44,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:53:44,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:53:44,080 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,080 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:53:44,080 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:53:44,080 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:53:44,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:53:44,080 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:53:44,081 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:44,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:53:44,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,081 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:44,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,081 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:53:44,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:53:44,082 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:44,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:53:44,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:44,083 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:44,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:44,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:53:44,084 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:44,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,085 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:44,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:44,085 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:44,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:44,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:44,090 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:44,090 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,090 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:44,090 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:53:44,090 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:53:44,090 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:53:44,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:53:44,090 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:53:44,090 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:53:44,090 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:53:44,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:53:44,090 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:53:44,090 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:53:44,090 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:44,090 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:53:44,090 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:53:44,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:53:44,090 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:53:44,091 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:53:44,091 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:44,091 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,091 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:53:44,091 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-05-10 04:53:44,091 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-10 04:53:44,091 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:53:44,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:53:44,091 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:44,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:53:44,091 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:44,091 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-05-10 04:53:44,092 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-10 04:53:44,091 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,092 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,192 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-10 04:53:44,193 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-10 04:53:44,216 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-10 04:53:44,218 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-05-10 04:53:44,218 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,218 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-05-10 04:53:44,218 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:53:44,219 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-05-10 04:53:44,219 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-05-10 04:53:44,219 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,219 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,227 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:53:44,227 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:44,227 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:53:44,227 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:53:44,227 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:53:44,227 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:53:44,227 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:53:44,227 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:44,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:53:44,227 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-10 04:53:44,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:53:44,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:53:44,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,228 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,228 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,228 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:53:44,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:53:44,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,229 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,229 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:53:44,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,229 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,229 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-05-10 04:53:44,229 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-05-10 04:53:44,229 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,229 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-05-10 04:53:44,229 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-05-10 04:53:44,229 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,229 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,230 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-05-10 04:53:44,230 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-05-10 04:53:44,230 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:53:44,230 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:53:44,231 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-05-10 04:53:44,230 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:53:44,231 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-05-10 04:53:44,232 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-10 04:53:44,232 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-10 04:53:44,232 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-05-10 04:53:44,232 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37995,1683694419499' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-05-10 04:53:44,232 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-05-10 04:53:44,232 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1683694422380.93c2cc329b1fe1893cfbc28c56960522. 2023-05-10 04:53:44,232 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-05-10 04:53:44,232 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36665,1683694419452' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-05-10 04:53:44,232 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1683694422380.93c2cc329b1fe1893cfbc28c56960522. started... 2023-05-10 04:53:44,232 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 93c2cc329b1fe1893cfbc28c56960522 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-10 04:53:44,234 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,235 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,235 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,235 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:53:44,238 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,239 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:53:44,258 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/.tmp/cf1/744bcdff675143b3b9b15d9f93d2e618 2023-05-10 04:53:44,288 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/.tmp/cf2/4ee8175dfe724214ad50b9f4f420f989 2023-05-10 04:53:44,298 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/.tmp/cf1/744bcdff675143b3b9b15d9f93d2e618 as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf1/744bcdff675143b3b9b15d9f93d2e618 2023-05-10 04:53:44,306 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf1/744bcdff675143b3b9b15d9f93d2e618, entries=10, sequenceid=45, filesize=5.0 K 2023-05-10 04:53:44,308 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/.tmp/cf2/4ee8175dfe724214ad50b9f4f420f989 as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf2/4ee8175dfe724214ad50b9f4f420f989 2023-05-10 04:53:44,316 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf2/4ee8175dfe724214ad50b9f4f420f989, entries=10, sequenceid=45, filesize=5.0 K 2023-05-10 04:53:44,317 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 93c2cc329b1fe1893cfbc28c56960522 in 85ms, sequenceid=45, compaction requested=true 2023-05-10 04:53:44,317 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 93c2cc329b1fe1893cfbc28c56960522: 2023-05-10 04:53:44,317 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1683694422380.93c2cc329b1fe1893cfbc28c56960522. 2023-05-10 04:53:44,317 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-10 04:53:44,319 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-10 04:53:44,319 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:44,319 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-05-10 04:53:44,319 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37823,1683694419534' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-05-10 04:53:44,329 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:44,329 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:44,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:53:44,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:53:44,331 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,331 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:53:44,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:53:44,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:53:44,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:44,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:44,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:44,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:53:44,333 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37823,1683694419534' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-05-10 04:53:44,333 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1119deb4[Count = 0] remaining members to acquire global barrier 2023-05-10 04:53:44,333 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-05-10 04:53:44,333 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,334 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,334 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,334 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,334 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,335 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:44,335 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-05-10 04:53:44,335 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-05-10 04:53:44,334 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,335 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,37995,1683694419499' in zk 2023-05-10 04:53:44,335 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,335 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-10 04:53:44,335 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-05-10 04:53:44,335 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-05-10 04:53:44,335 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-05-10 04:53:44,335 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,37823,1683694419534' in zk 2023-05-10 04:53:44,335 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-05-10 04:53:44,335 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,36665,1683694419452' in zk 2023-05-10 04:53:44,336 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-05-10 04:53:44,336 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:53:44,337 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-05-10 04:53:44,338 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:44,338 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-05-10 04:53:44,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:44,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:53:44,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:53:44,338 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:53:44,339 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-05-10 04:53:44,340 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-05-10 04:53:44,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:53:44,340 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:53:44,340 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-05-10 04:53:44,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:53:44,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:44,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:44,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:44,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:53:44,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:44,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:44,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:44,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,37823,1683694419534': 2023-05-10 04:53:44,344 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37823,1683694419534' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-05-10 04:53:44,344 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-05-10 04:53:44,344 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-10 04:53:44,344 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-10 04:53:44,344 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,344 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-10 04:53:44,346 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,346 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,346 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,346 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,346 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:53:44,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,346 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:53:44,346 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,346 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,346 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,347 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:53:44,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:53:44,346 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:53:44,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:53:44,347 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:53:44,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:53:44,347 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:44,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:53:44,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,348 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:53:44,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:53:44,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:53:44,348 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:44,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,348 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:44,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:53:44,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:44,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:44,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:44,351 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:44,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:53:44,351 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:44,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,351 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:44,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:44,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:44,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:44,354 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:44,355 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,355 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:53:44,355 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:53:44,355 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:44,355 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:53:44,355 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:53:44,355 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:53:44,355 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-05-10 04:53:44,355 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:53:44,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:53:44,355 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-10 04:53:44,355 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:53:44,355 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:53:44,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:53:44,355 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:44,355 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,355 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:53:44,355 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:53:44,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:53:44,356 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:53:44,356 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:44,356 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-05-10 04:53:44,357 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-10 04:53:44,357 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:53:44,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:53:44,356 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:53:44,357 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:53:44,356 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:53:44,357 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:44,357 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:44,357 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,357 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,457 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-10 04:53:44,459 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-10 04:53:44,481 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-10 04:53:44,483 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-05-10 04:53:44,483 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,483 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-05-10 04:53:44,483 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:53:44,484 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-05-10 04:53:44,484 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-05-10 04:53:44,484 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,484 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,486 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:53:44,486 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:53:44,486 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:53:44,486 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:53:44,486 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:44,486 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:53:44,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:53:44,486 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:53:44,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:53:44,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:53:44,486 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:44,487 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-10 04:53:44,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,487 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,487 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,487 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:53:44,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:53:44,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:53:44,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,488 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,488 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,488 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,488 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,488 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-05-10 04:53:44,488 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-05-10 04:53:44,488 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-05-10 04:53:44,488 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,488 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,488 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-05-10 04:53:44,488 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-05-10 04:53:44,488 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,488 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:53:44,489 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-05-10 04:53:44,488 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:53:44,489 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:53:44,489 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-05-10 04:53:44,492 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-05-10 04:53:44,490 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-05-10 04:53:44,492 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-05-10 04:53:44,492 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37995,1683694419499' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-05-10 04:53:44,492 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-05-10 04:53:44,492 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36665,1683694419452' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-05-10 04:53:44,492 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1683694422380.93c2cc329b1fe1893cfbc28c56960522. 2023-05-10 04:53:44,492 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-10 04:53:44,492 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-10 04:53:44,492 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1683694422380.93c2cc329b1fe1893cfbc28c56960522. started... 2023-05-10 04:53:44,492 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 93c2cc329b1fe1893cfbc28c56960522 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-10 04:53:44,495 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,496 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,496 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,496 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:53:44,496 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,496 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:53:44,511 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/.tmp/cf1/e4b432d539b1457f9648cdf383c363c9 2023-05-10 04:53:44,536 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/.tmp/cf2/a4dcfc1811054c03bee550c410a35fbb 2023-05-10 04:53:44,545 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/.tmp/cf1/e4b432d539b1457f9648cdf383c363c9 as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf1/e4b432d539b1457f9648cdf383c363c9 2023-05-10 04:53:44,552 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf1/e4b432d539b1457f9648cdf383c363c9, entries=10, sequenceid=58, filesize=5.0 K 2023-05-10 04:53:44,553 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/.tmp/cf2/a4dcfc1811054c03bee550c410a35fbb as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf2/a4dcfc1811054c03bee550c410a35fbb 2023-05-10 04:53:44,561 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf2/a4dcfc1811054c03bee550c410a35fbb, entries=10, sequenceid=58, filesize=5.0 K 2023-05-10 04:53:44,562 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 93c2cc329b1fe1893cfbc28c56960522 in 70ms, sequenceid=58, compaction requested=true 2023-05-10 04:53:44,562 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 93c2cc329b1fe1893cfbc28c56960522: 2023-05-10 04:53:44,562 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1683694422380.93c2cc329b1fe1893cfbc28c56960522. 2023-05-10 04:53:44,562 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-10 04:53:44,563 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-10 04:53:44,563 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:44,563 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-05-10 04:53:44,563 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37823,1683694419534' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-05-10 04:53:44,565 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:44,565 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:44,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:53:44,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:53:44,566 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,566 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:53:44,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:53:44,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:53:44,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:44,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:44,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:44,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:53:44,568 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37823,1683694419534' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-05-10 04:53:44,569 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-05-10 04:53:44,569 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@30165818[Count = 0] remaining members to acquire global barrier 2023-05-10 04:53:44,569 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,570 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,570 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,570 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,570 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,570 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,570 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,570 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-05-10 04:53:44,570 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-05-10 04:53:44,570 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-05-10 04:53:44,570 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-05-10 04:53:44,570 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:44,571 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-10 04:53:44,571 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,36665,1683694419452' in zk 2023-05-10 04:53:44,571 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,37823,1683694419534' in zk 2023-05-10 04:53:44,570 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-05-10 04:53:44,571 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-05-10 04:53:44,571 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,37995,1683694419499' in zk 2023-05-10 04:53:44,573 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-05-10 04:53:44,574 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:53:44,574 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-05-10 04:53:44,575 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:44,575 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-05-10 04:53:44,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:44,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:53:44,575 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:53:44,575 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-05-10 04:53:44,575 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-05-10 04:53:44,575 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:53:44,575 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:53:44,576 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-05-10 04:53:44,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:53:44,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:53:44,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:44,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:44,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:44,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:53:44,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:44,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:44,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:44,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,37823,1683694419534': 2023-05-10 04:53:44,581 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37823,1683694419534' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-05-10 04:53:44,581 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-05-10 04:53:44,581 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-10 04:53:44,581 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-10 04:53:44,581 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,581 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-10 04:53:44,582 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,583 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,583 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,583 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,583 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:53:44,583 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:53:44,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:53:44,583 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,583 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:53:44,583 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,583 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:53:44,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:53:44,583 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:53:44,583 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:44,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:53:44,584 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:53:44,584 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:53:44,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:53:44,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:53:44,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,584 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:44,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:53:44,584 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:44,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:44,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:44,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:44,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:53:44,586 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:44,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,587 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:44,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:44,587 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:44,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:44,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:44,594 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:44,594 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,594 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:53:44,594 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:53:44,594 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:53:44,595 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:53:44,595 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:44,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:53:44,595 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:44,595 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:53:44,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:53:44,595 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-05-10 04:53:44,595 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-10 04:53:44,595 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:53:44,594 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:53:44,595 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:53:44,595 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:53:44,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:53:44,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:53:44,595 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:53:44,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:53:44,595 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:53:44,595 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:53:44,595 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,597 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-05-10 04:53:44,597 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:53:44,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:53:44,597 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:44,597 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,597 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:44,597 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:44,597 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,597 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,597 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-10 04:53:44,697 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-10 04:53:44,699 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-10 04:53:44,719 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-10 04:53:44,722 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-05-10 04:53:44,722 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,722 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-05-10 04:53:44,722 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:53:44,723 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-05-10 04:53:44,723 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-05-10 04:53:44,723 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,723 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,725 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:53:44,725 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:44,725 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:53:44,725 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:53:44,725 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:53:44,725 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:53:44,725 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:53:44,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:53:44,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:53:44,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:53:44,725 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:44,725 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-10 04:53:44,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,725 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,726 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,726 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,726 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:53:44,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:53:44,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:53:44,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,726 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,726 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,727 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,727 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,727 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-05-10 04:53:44,727 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-05-10 04:53:44,727 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,727 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-05-10 04:53:44,727 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-05-10 04:53:44,727 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,727 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:53:44,727 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-05-10 04:53:44,728 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-05-10 04:53:44,727 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,728 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-05-10 04:53:44,728 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:53:44,728 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-05-10 04:53:44,728 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-05-10 04:53:44,728 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36665,1683694419452' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-05-10 04:53:44,729 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:53:44,729 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-10 04:53:44,729 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-10 04:53:44,729 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-05-10 04:53:44,729 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-05-10 04:53:44,729 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1683694422380.93c2cc329b1fe1893cfbc28c56960522. 2023-05-10 04:53:44,729 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37995,1683694419499' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-05-10 04:53:44,729 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1683694422380.93c2cc329b1fe1893cfbc28c56960522. started... 2023-05-10 04:53:44,730 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 93c2cc329b1fe1893cfbc28c56960522 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-10 04:53:44,731 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,732 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,732 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:53:44,732 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,732 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,733 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:53:44,748 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/.tmp/cf1/a9634e7ee1844806a05a96cd8e89dd4d 2023-05-10 04:53:44,771 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/.tmp/cf2/7a78adce76124917861cfd93394e25fd 2023-05-10 04:53:44,779 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/.tmp/cf1/a9634e7ee1844806a05a96cd8e89dd4d as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf1/a9634e7ee1844806a05a96cd8e89dd4d 2023-05-10 04:53:44,787 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf1/a9634e7ee1844806a05a96cd8e89dd4d, entries=10, sequenceid=71, filesize=5.0 K 2023-05-10 04:53:44,788 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/.tmp/cf2/7a78adce76124917861cfd93394e25fd as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf2/7a78adce76124917861cfd93394e25fd 2023-05-10 04:53:44,795 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf2/7a78adce76124917861cfd93394e25fd, entries=10, sequenceid=71, filesize=5.0 K 2023-05-10 04:53:44,797 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 93c2cc329b1fe1893cfbc28c56960522 in 67ms, sequenceid=71, compaction requested=true 2023-05-10 04:53:44,797 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 93c2cc329b1fe1893cfbc28c56960522: 2023-05-10 04:53:44,797 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1683694422380.93c2cc329b1fe1893cfbc28c56960522. 2023-05-10 04:53:44,797 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-10 04:53:44,798 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-10 04:53:44,798 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:44,798 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-05-10 04:53:44,798 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37823,1683694419534' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-05-10 04:53:44,801 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,801 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:44,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:44,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:53:44,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:53:44,802 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,802 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:53:44,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:53:44,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:53:44,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:44,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:44,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:44,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:53:44,804 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37823,1683694419534' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-05-10 04:53:44,804 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@39b57c37[Count = 0] remaining members to acquire global barrier 2023-05-10 04:53:44,804 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-05-10 04:53:44,804 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,805 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,805 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,806 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,805 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,806 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,806 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,806 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:44,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,806 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-05-10 04:53:44,806 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-05-10 04:53:44,806 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-05-10 04:53:44,806 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-05-10 04:53:44,806 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-10 04:53:44,806 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-05-10 04:53:44,806 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,37995,1683694419499' in zk 2023-05-10 04:53:44,806 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-05-10 04:53:44,806 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,36665,1683694419452' in zk 2023-05-10 04:53:44,806 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase4.apache.org,37823,1683694419534' in zk 2023-05-10 04:53:44,808 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-05-10 04:53:44,808 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:53:44,808 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-05-10 04:53:44,809 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-05-10 04:53:44,809 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:44,809 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:53:44,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:44,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:53:44,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:53:44,809 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-05-10 04:53:44,809 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-05-10 04:53:44,810 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:53:44,810 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-05-10 04:53:44,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:53:44,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:53:44,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:44,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:44,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:44,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:53:44,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:44,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:44,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:44,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase4.apache.org,37823,1683694419534': 2023-05-10 04:53:44,815 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37823,1683694419534' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-05-10 04:53:44,815 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-05-10 04:53:44,815 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-10 04:53:44,815 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-10 04:53:44,815 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,815 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-10 04:53:44,818 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,818 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:53:44,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:53:44,818 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,818 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,818 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:53:44,818 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,818 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:53:44,818 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:53:44,818 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,818 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:53:44,818 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:44,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,819 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:44,819 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:53:44,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:53:44,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:53:44,819 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:53:44,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:53:44,819 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:53:44,819 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:44,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:53:44,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:44,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:44,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:44,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:53:44,821 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:44,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,822 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:44,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:44,822 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:44,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:44,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:44,825 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:44,825 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:53:44,825 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:53:44,825 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:53:44,825 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:53:44,825 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:53:44,825 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,825 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:53:44,826 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:44,826 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:53:44,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:53:44,825 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-05-10 04:53:44,826 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-10 04:53:44,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:53:44,825 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:53:44,825 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:53:44,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:53:44,826 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:44,826 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:53:44,827 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,827 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:53:44,827 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:53:44,827 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-05-10 04:53:44,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:53:44,827 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:44,827 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:53:44,827 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-10 04:53:44,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:53:44,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:53:44,827 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,828 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:44,828 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:44,828 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,828 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-05-10 04:53:44,927 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-10 04:53:44,929 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-10 04:53:44,955 DEBUG [Listener at localhost/45007] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:44,961 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(1029): Stored pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=93c2cc329b1fe1893cfbc28c56960522, daughterA=19040b6538a6bdeed7c833cfb44614b6, daughterB=054f69c06f126aad97b290410d5d6188 2023-05-10 04:53:44,963 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=93c2cc329b1fe1893cfbc28c56960522, daughterA=19040b6538a6bdeed7c833cfb44614b6, daughterB=054f69c06f126aad97b290410d5d6188 2023-05-10 04:53:44,963 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=93c2cc329b1fe1893cfbc28c56960522, daughterA=19040b6538a6bdeed7c833cfb44614b6, daughterB=054f69c06f126aad97b290410d5d6188 2023-05-10 04:53:44,963 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=93c2cc329b1fe1893cfbc28c56960522, daughterA=19040b6538a6bdeed7c833cfb44614b6, daughterB=054f69c06f126aad97b290410d5d6188 2023-05-10 04:53:44,970 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=93c2cc329b1fe1893cfbc28c56960522, UNASSIGN}] 2023-05-10 04:53:44,971 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=93c2cc329b1fe1893cfbc28c56960522, UNASSIGN 2023-05-10 04:53:44,972 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=93c2cc329b1fe1893cfbc28c56960522, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:44,972 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1683694422380.93c2cc329b1fe1893cfbc28c56960522.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1683694424972"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683694424972"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683694424972"}]},"ts":"1683694424972"} 2023-05-10 04:53:44,974 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=14, ppid=13, state=RUNNABLE; CloseRegionProcedure 93c2cc329b1fe1893cfbc28c56960522, server=jenkins-hbase4.apache.org,37823,1683694419534}] 2023-05-10 04:53:45,134 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 93c2cc329b1fe1893cfbc28c56960522 2023-05-10 04:53:45,135 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 93c2cc329b1fe1893cfbc28c56960522, disabling compactions & flushes 2023-05-10 04:53:45,135 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionUnevenDaughter,,1683694422380.93c2cc329b1fe1893cfbc28c56960522. 2023-05-10 04:53:45,135 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,,1683694422380.93c2cc329b1fe1893cfbc28c56960522. 2023-05-10 04:53:45,135 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionUnevenDaughter,,1683694422380.93c2cc329b1fe1893cfbc28c56960522. after waiting 0 ms 2023-05-10 04:53:45,135 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionUnevenDaughter,,1683694422380.93c2cc329b1fe1893cfbc28c56960522. 2023-05-10 04:53:45,135 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 93c2cc329b1fe1893cfbc28c56960522 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-05-10 04:53:45,152 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/.tmp/cf1/427cc49d3a8e4682b202f4b0673d8806 2023-05-10 04:53:45,177 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/.tmp/cf2/48782fe6bd7d4d8996815ff75cdfac23 2023-05-10 04:53:45,185 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/.tmp/cf1/427cc49d3a8e4682b202f4b0673d8806 as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf1/427cc49d3a8e4682b202f4b0673d8806 2023-05-10 04:53:45,192 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf1/427cc49d3a8e4682b202f4b0673d8806, entries=8, sequenceid=82, filesize=5.0 K 2023-05-10 04:53:45,194 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/.tmp/cf2/48782fe6bd7d4d8996815ff75cdfac23 as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf2/48782fe6bd7d4d8996815ff75cdfac23 2023-05-10 04:53:45,203 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf2/48782fe6bd7d4d8996815ff75cdfac23, entries=8, sequenceid=82, filesize=5.0 K 2023-05-10 04:53:45,204 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 93c2cc329b1fe1893cfbc28c56960522 in 69ms, sequenceid=82, compaction requested=true 2023-05-10 04:53:45,227 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-05-10 04:53:45,228 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-10 04:53:45,229 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionUnevenDaughter,,1683694422380.93c2cc329b1fe1893cfbc28c56960522. 2023-05-10 04:53:45,229 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 93c2cc329b1fe1893cfbc28c56960522: 2023-05-10 04:53:45,232 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 93c2cc329b1fe1893cfbc28c56960522 2023-05-10 04:53:45,232 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=93c2cc329b1fe1893cfbc28c56960522, regionState=CLOSED 2023-05-10 04:53:45,233 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,,1683694422380.93c2cc329b1fe1893cfbc28c56960522.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1683694425232"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683694425232"}]},"ts":"1683694425232"} 2023-05-10 04:53:45,237 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=14, resume processing ppid=13 2023-05-10 04:53:45,238 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=14, ppid=13, state=SUCCESS; CloseRegionProcedure 93c2cc329b1fe1893cfbc28c56960522, server=jenkins-hbase4.apache.org,37823,1683694419534 in 261 msec 2023-05-10 04:53:45,240 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=13, resume processing ppid=12 2023-05-10 04:53:45,240 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=13, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=93c2cc329b1fe1893cfbc28c56960522, UNASSIGN in 268 msec 2023-05-10 04:53:45,259 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=12 splitting 14 storefiles, region=93c2cc329b1fe1893cfbc28c56960522, threads=14 2023-05-10 04:53:45,260 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf2/48782fe6bd7d4d8996815ff75cdfac23 for region: 93c2cc329b1fe1893cfbc28c56960522 2023-05-10 04:53:45,260 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf2/4ee8175dfe724214ad50b9f4f420f989 for region: 93c2cc329b1fe1893cfbc28c56960522 2023-05-10 04:53:45,260 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf2/76f6f9416941403596b6ff3caccb861c for region: 93c2cc329b1fe1893cfbc28c56960522 2023-05-10 04:53:45,260 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf2/7a78adce76124917861cfd93394e25fd for region: 93c2cc329b1fe1893cfbc28c56960522 2023-05-10 04:53:45,260 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf2/a4dcfc1811054c03bee550c410a35fbb for region: 93c2cc329b1fe1893cfbc28c56960522 2023-05-10 04:53:45,261 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf2/b1cbcf3117244e049435fa8e701a7a80 for region: 93c2cc329b1fe1893cfbc28c56960522 2023-05-10 04:53:45,261 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf1/6fc2425571254b1b8437fabc5c780c71 for region: 93c2cc329b1fe1893cfbc28c56960522 2023-05-10 04:53:45,261 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf1/427cc49d3a8e4682b202f4b0673d8806 for region: 93c2cc329b1fe1893cfbc28c56960522 2023-05-10 04:53:45,261 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf2/bd68bbeaf0094fd0a415560924da2bce for region: 93c2cc329b1fe1893cfbc28c56960522 2023-05-10 04:53:45,262 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf1/744bcdff675143b3b9b15d9f93d2e618 for region: 93c2cc329b1fe1893cfbc28c56960522 2023-05-10 04:53:45,262 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf1/a9634e7ee1844806a05a96cd8e89dd4d for region: 93c2cc329b1fe1893cfbc28c56960522 2023-05-10 04:53:45,263 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf1/9333fac35a6e43f2ae1eb05c2a8c43e2 for region: 93c2cc329b1fe1893cfbc28c56960522 2023-05-10 04:53:45,263 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf1/be24ca36992745b7a1a3b25d9bc68d53 for region: 93c2cc329b1fe1893cfbc28c56960522 2023-05-10 04:53:45,263 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf1/e4b432d539b1457f9648cdf383c363c9 for region: 93c2cc329b1fe1893cfbc28c56960522 2023-05-10 04:53:45,342 DEBUG [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf2/48782fe6bd7d4d8996815ff75cdfac23, top=true 2023-05-10 04:53:45,348 DEBUG [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf1/a9634e7ee1844806a05a96cd8e89dd4d, top=true 2023-05-10 04:53:45,348 DEBUG [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf2/7a78adce76124917861cfd93394e25fd, top=true 2023-05-10 04:53:45,351 DEBUG [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf1/e4b432d539b1457f9648cdf383c363c9, top=true 2023-05-10 04:53:45,352 DEBUG [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf2/a4dcfc1811054c03bee550c410a35fbb, top=true 2023-05-10 04:53:45,352 DEBUG [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf1/427cc49d3a8e4682b202f4b0673d8806, top=true 2023-05-10 04:53:45,415 INFO [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/.splits/054f69c06f126aad97b290410d5d6188/cf1/testSplitTableRegionUnevenDaughter=93c2cc329b1fe1893cfbc28c56960522-427cc49d3a8e4682b202f4b0673d8806 for child: 054f69c06f126aad97b290410d5d6188, parent: 93c2cc329b1fe1893cfbc28c56960522 2023-05-10 04:53:45,415 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf1/427cc49d3a8e4682b202f4b0673d8806 for region: 93c2cc329b1fe1893cfbc28c56960522 2023-05-10 04:53:45,419 INFO [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/.splits/054f69c06f126aad97b290410d5d6188/cf2/testSplitTableRegionUnevenDaughter=93c2cc329b1fe1893cfbc28c56960522-a4dcfc1811054c03bee550c410a35fbb for child: 054f69c06f126aad97b290410d5d6188, parent: 93c2cc329b1fe1893cfbc28c56960522 2023-05-10 04:53:45,419 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf2/a4dcfc1811054c03bee550c410a35fbb for region: 93c2cc329b1fe1893cfbc28c56960522 2023-05-10 04:53:45,423 INFO [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/.splits/054f69c06f126aad97b290410d5d6188/cf2/testSplitTableRegionUnevenDaughter=93c2cc329b1fe1893cfbc28c56960522-48782fe6bd7d4d8996815ff75cdfac23 for child: 054f69c06f126aad97b290410d5d6188, parent: 93c2cc329b1fe1893cfbc28c56960522 2023-05-10 04:53:45,423 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf2/48782fe6bd7d4d8996815ff75cdfac23 for region: 93c2cc329b1fe1893cfbc28c56960522 2023-05-10 04:53:45,423 INFO [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/.splits/054f69c06f126aad97b290410d5d6188/cf1/testSplitTableRegionUnevenDaughter=93c2cc329b1fe1893cfbc28c56960522-a9634e7ee1844806a05a96cd8e89dd4d for child: 054f69c06f126aad97b290410d5d6188, parent: 93c2cc329b1fe1893cfbc28c56960522 2023-05-10 04:53:45,423 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf1/a9634e7ee1844806a05a96cd8e89dd4d for region: 93c2cc329b1fe1893cfbc28c56960522 2023-05-10 04:53:45,438 INFO [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/.splits/054f69c06f126aad97b290410d5d6188/cf2/testSplitTableRegionUnevenDaughter=93c2cc329b1fe1893cfbc28c56960522-7a78adce76124917861cfd93394e25fd for child: 054f69c06f126aad97b290410d5d6188, parent: 93c2cc329b1fe1893cfbc28c56960522 2023-05-10 04:53:45,438 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf2/7a78adce76124917861cfd93394e25fd for region: 93c2cc329b1fe1893cfbc28c56960522 2023-05-10 04:53:45,455 INFO [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/.splits/054f69c06f126aad97b290410d5d6188/cf1/testSplitTableRegionUnevenDaughter=93c2cc329b1fe1893cfbc28c56960522-e4b432d539b1457f9648cdf383c363c9 for child: 054f69c06f126aad97b290410d5d6188, parent: 93c2cc329b1fe1893cfbc28c56960522 2023-05-10 04:53:45,455 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf1/e4b432d539b1457f9648cdf383c363c9 for region: 93c2cc329b1fe1893cfbc28c56960522 2023-05-10 04:53:45,511 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf2/4ee8175dfe724214ad50b9f4f420f989 for region: 93c2cc329b1fe1893cfbc28c56960522 2023-05-10 04:53:45,544 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf2/76f6f9416941403596b6ff3caccb861c for region: 93c2cc329b1fe1893cfbc28c56960522 2023-05-10 04:53:45,551 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf1/9333fac35a6e43f2ae1eb05c2a8c43e2 for region: 93c2cc329b1fe1893cfbc28c56960522 2023-05-10 04:53:45,570 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf1/744bcdff675143b3b9b15d9f93d2e618 for region: 93c2cc329b1fe1893cfbc28c56960522 2023-05-10 04:53:45,572 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf1/be24ca36992745b7a1a3b25d9bc68d53 for region: 93c2cc329b1fe1893cfbc28c56960522 2023-05-10 04:53:45,576 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf2/bd68bbeaf0094fd0a415560924da2bce for region: 93c2cc329b1fe1893cfbc28c56960522 2023-05-10 04:53:45,577 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf2/b1cbcf3117244e049435fa8e701a7a80 for region: 93c2cc329b1fe1893cfbc28c56960522 2023-05-10 04:53:45,579 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf1/6fc2425571254b1b8437fabc5c780c71 for region: 93c2cc329b1fe1893cfbc28c56960522 2023-05-10 04:53:45,579 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=12 split storefiles for region 93c2cc329b1fe1893cfbc28c56960522 Daughter A: 8 storefiles, Daughter B: 14 storefiles. 2023-05-10 04:53:45,632 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/19040b6538a6bdeed7c833cfb44614b6/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-05-10 04:53:45,634 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/054f69c06f126aad97b290410d5d6188/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-05-10 04:53:45,637 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1683694422380.93c2cc329b1fe1893cfbc28c56960522.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1683694425637"},{"qualifier":"splitA","vlen":70,"tag":[],"timestamp":"1683694425637"},{"qualifier":"splitB","vlen":70,"tag":[],"timestamp":"1683694425637"}]},"ts":"1683694425637"} 2023-05-10 04:53:45,637 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1683694424955.19040b6538a6bdeed7c833cfb44614b6.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1683694425637"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683694425637"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683694425637"}]},"ts":"1683694425637"} 2023-05-10 04:53:45,637 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,26,1683694424955.054f69c06f126aad97b290410d5d6188.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1683694425637"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683694425637"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683694425637"}]},"ts":"1683694425637"} 2023-05-10 04:53:45,695 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=19040b6538a6bdeed7c833cfb44614b6, ASSIGN}, {pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=054f69c06f126aad97b290410d5d6188, ASSIGN}] 2023-05-10 04:53:45,696 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=054f69c06f126aad97b290410d5d6188, ASSIGN 2023-05-10 04:53:45,697 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=054f69c06f126aad97b290410d5d6188, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,37823,1683694419534; forceNewPlan=false, retain=false 2023-05-10 04:53:45,698 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=19040b6538a6bdeed7c833cfb44614b6, ASSIGN 2023-05-10 04:53:45,699 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=19040b6538a6bdeed7c833cfb44614b6, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,37823,1683694419534; forceNewPlan=false, retain=false 2023-05-10 04:53:45,847 INFO [jenkins-hbase4:37903] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-05-10 04:53:45,849 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=16 updating hbase:meta row=054f69c06f126aad97b290410d5d6188, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:45,849 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,26,1683694424955.054f69c06f126aad97b290410d5d6188.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1683694425849"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683694425849"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683694425849"}]},"ts":"1683694425849"} 2023-05-10 04:53:45,851 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=17, ppid=16, state=RUNNABLE; OpenRegionProcedure 054f69c06f126aad97b290410d5d6188, server=jenkins-hbase4.apache.org,37823,1683694419534}] 2023-05-10 04:53:45,855 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=19040b6538a6bdeed7c833cfb44614b6, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:45,856 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1683694424955.19040b6538a6bdeed7c833cfb44614b6.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1683694425855"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683694425855"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683694425855"}]},"ts":"1683694425855"} 2023-05-10 04:53:45,857 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=18, ppid=15, state=RUNNABLE; OpenRegionProcedure 19040b6538a6bdeed7c833cfb44614b6, server=jenkins-hbase4.apache.org,37823,1683694419534}] 2023-05-10 04:53:46,008 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionUnevenDaughter,,1683694424955.19040b6538a6bdeed7c833cfb44614b6. 2023-05-10 04:53:46,009 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 19040b6538a6bdeed7c833cfb44614b6, NAME => 'testSplitTableRegionUnevenDaughter,,1683694424955.19040b6538a6bdeed7c833cfb44614b6.', STARTKEY => '', ENDKEY => '26'} 2023-05-10 04:53:46,009 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-05-10 04:53:46,009 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionUnevenDaughter 19040b6538a6bdeed7c833cfb44614b6 2023-05-10 04:53:46,009 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionUnevenDaughter,,1683694424955.19040b6538a6bdeed7c833cfb44614b6.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-10 04:53:46,010 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 19040b6538a6bdeed7c833cfb44614b6 2023-05-10 04:53:46,010 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 19040b6538a6bdeed7c833cfb44614b6 2023-05-10 04:53:46,011 INFO [StoreOpener-19040b6538a6bdeed7c833cfb44614b6-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 19040b6538a6bdeed7c833cfb44614b6 2023-05-10 04:53:46,013 DEBUG [StoreOpener-19040b6538a6bdeed7c833cfb44614b6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/19040b6538a6bdeed7c833cfb44614b6/cf1 2023-05-10 04:53:46,013 DEBUG [StoreOpener-19040b6538a6bdeed7c833cfb44614b6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/19040b6538a6bdeed7c833cfb44614b6/cf1 2023-05-10 04:53:46,013 INFO [StoreOpener-19040b6538a6bdeed7c833cfb44614b6-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 19040b6538a6bdeed7c833cfb44614b6 columnFamilyName cf1 2023-05-10 04:53:46,051 DEBUG [StoreOpener-19040b6538a6bdeed7c833cfb44614b6-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/19040b6538a6bdeed7c833cfb44614b6/cf1/6fc2425571254b1b8437fabc5c780c71.93c2cc329b1fe1893cfbc28c56960522->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf1/6fc2425571254b1b8437fabc5c780c71-bottom 2023-05-10 04:53:46,059 DEBUG [StoreOpener-19040b6538a6bdeed7c833cfb44614b6-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/19040b6538a6bdeed7c833cfb44614b6/cf1/744bcdff675143b3b9b15d9f93d2e618.93c2cc329b1fe1893cfbc28c56960522->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf1/744bcdff675143b3b9b15d9f93d2e618-bottom 2023-05-10 04:53:46,066 DEBUG [StoreOpener-19040b6538a6bdeed7c833cfb44614b6-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/19040b6538a6bdeed7c833cfb44614b6/cf1/9333fac35a6e43f2ae1eb05c2a8c43e2.93c2cc329b1fe1893cfbc28c56960522->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf1/9333fac35a6e43f2ae1eb05c2a8c43e2-bottom 2023-05-10 04:53:46,074 DEBUG [StoreOpener-19040b6538a6bdeed7c833cfb44614b6-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/19040b6538a6bdeed7c833cfb44614b6/cf1/be24ca36992745b7a1a3b25d9bc68d53.93c2cc329b1fe1893cfbc28c56960522->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf1/be24ca36992745b7a1a3b25d9bc68d53-bottom 2023-05-10 04:53:46,074 INFO [StoreOpener-19040b6538a6bdeed7c833cfb44614b6-1] regionserver.HStore(310): Store=19040b6538a6bdeed7c833cfb44614b6/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-10 04:53:46,075 INFO [StoreOpener-19040b6538a6bdeed7c833cfb44614b6-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 19040b6538a6bdeed7c833cfb44614b6 2023-05-10 04:53:46,076 DEBUG [StoreOpener-19040b6538a6bdeed7c833cfb44614b6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/19040b6538a6bdeed7c833cfb44614b6/cf2 2023-05-10 04:53:46,076 DEBUG [StoreOpener-19040b6538a6bdeed7c833cfb44614b6-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/19040b6538a6bdeed7c833cfb44614b6/cf2 2023-05-10 04:53:46,076 INFO [StoreOpener-19040b6538a6bdeed7c833cfb44614b6-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 19040b6538a6bdeed7c833cfb44614b6 columnFamilyName cf2 2023-05-10 04:53:46,092 DEBUG [StoreOpener-19040b6538a6bdeed7c833cfb44614b6-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/19040b6538a6bdeed7c833cfb44614b6/cf2/4ee8175dfe724214ad50b9f4f420f989.93c2cc329b1fe1893cfbc28c56960522->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf2/4ee8175dfe724214ad50b9f4f420f989-bottom 2023-05-10 04:53:46,098 DEBUG [StoreOpener-19040b6538a6bdeed7c833cfb44614b6-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/19040b6538a6bdeed7c833cfb44614b6/cf2/76f6f9416941403596b6ff3caccb861c.93c2cc329b1fe1893cfbc28c56960522->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf2/76f6f9416941403596b6ff3caccb861c-bottom 2023-05-10 04:53:46,105 DEBUG [StoreOpener-19040b6538a6bdeed7c833cfb44614b6-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/19040b6538a6bdeed7c833cfb44614b6/cf2/b1cbcf3117244e049435fa8e701a7a80.93c2cc329b1fe1893cfbc28c56960522->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf2/b1cbcf3117244e049435fa8e701a7a80-bottom 2023-05-10 04:53:46,112 DEBUG [StoreOpener-19040b6538a6bdeed7c833cfb44614b6-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/19040b6538a6bdeed7c833cfb44614b6/cf2/bd68bbeaf0094fd0a415560924da2bce.93c2cc329b1fe1893cfbc28c56960522->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf2/bd68bbeaf0094fd0a415560924da2bce-bottom 2023-05-10 04:53:46,112 INFO [StoreOpener-19040b6538a6bdeed7c833cfb44614b6-1] regionserver.HStore(310): Store=19040b6538a6bdeed7c833cfb44614b6/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-10 04:53:46,113 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/19040b6538a6bdeed7c833cfb44614b6 2023-05-10 04:53:46,115 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/19040b6538a6bdeed7c833cfb44614b6 2023-05-10 04:53:46,117 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionUnevenDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-05-10 04:53:46,119 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 19040b6538a6bdeed7c833cfb44614b6 2023-05-10 04:53:46,120 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 19040b6538a6bdeed7c833cfb44614b6; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72978606, jitterRate=0.08746597170829773}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-05-10 04:53:46,120 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 19040b6538a6bdeed7c833cfb44614b6: 2023-05-10 04:53:46,122 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2334): Post open deploy tasks for testSplitTableRegionUnevenDaughter,,1683694424955.19040b6538a6bdeed7c833cfb44614b6., pid=18, masterSystemTime=1683694426003 2023-05-10 04:53:46,124 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-10 04:53:46,125 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-05-10 04:53:46,126 DEBUG [RS:2;jenkins-hbase4:37823-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-05-10 04:53:46,126 DEBUG [RS:2;jenkins-hbase4:37823-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-05-10 04:53:46,128 INFO [RS:2;jenkins-hbase4:37823-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,,1683694424955.19040b6538a6bdeed7c833cfb44614b6. 2023-05-10 04:53:46,128 INFO [RS:2;jenkins-hbase4:37823-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,,1683694424955.19040b6538a6bdeed7c833cfb44614b6. 2023-05-10 04:53:46,128 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2361): Finished post open deploy task for testSplitTableRegionUnevenDaughter,,1683694424955.19040b6538a6bdeed7c833cfb44614b6. 2023-05-10 04:53:46,128 DEBUG [RS:2;jenkins-hbase4:37823-shortCompactions-0] regionserver.HStore(1912): 19040b6538a6bdeed7c833cfb44614b6/cf1 is initiating minor compaction (all files) 2023-05-10 04:53:46,128 DEBUG [RS:2;jenkins-hbase4:37823-longCompactions-0] regionserver.HStore(1912): 19040b6538a6bdeed7c833cfb44614b6/cf2 is initiating minor compaction (all files) 2023-05-10 04:53:46,128 INFO [RS:2;jenkins-hbase4:37823-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 19040b6538a6bdeed7c833cfb44614b6/cf1 in testSplitTableRegionUnevenDaughter,,1683694424955.19040b6538a6bdeed7c833cfb44614b6. 2023-05-10 04:53:46,128 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionUnevenDaughter,,1683694424955.19040b6538a6bdeed7c833cfb44614b6. 2023-05-10 04:53:46,128 INFO [RS:2;jenkins-hbase4:37823-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 19040b6538a6bdeed7c833cfb44614b6/cf2 in testSplitTableRegionUnevenDaughter,,1683694424955.19040b6538a6bdeed7c833cfb44614b6. 2023-05-10 04:53:46,129 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionUnevenDaughter,26,1683694424955.054f69c06f126aad97b290410d5d6188. 2023-05-10 04:53:46,129 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=19040b6538a6bdeed7c833cfb44614b6, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:46,129 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 054f69c06f126aad97b290410d5d6188, NAME => 'testSplitTableRegionUnevenDaughter,26,1683694424955.054f69c06f126aad97b290410d5d6188.', STARTKEY => '26', ENDKEY => ''} 2023-05-10 04:53:46,129 INFO [RS:2;jenkins-hbase4:37823-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/19040b6538a6bdeed7c833cfb44614b6/cf1/9333fac35a6e43f2ae1eb05c2a8c43e2.93c2cc329b1fe1893cfbc28c56960522->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf1/9333fac35a6e43f2ae1eb05c2a8c43e2-bottom, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/19040b6538a6bdeed7c833cfb44614b6/cf1/6fc2425571254b1b8437fabc5c780c71.93c2cc329b1fe1893cfbc28c56960522->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf1/6fc2425571254b1b8437fabc5c780c71-bottom, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/19040b6538a6bdeed7c833cfb44614b6/cf1/be24ca36992745b7a1a3b25d9bc68d53.93c2cc329b1fe1893cfbc28c56960522->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf1/be24ca36992745b7a1a3b25d9bc68d53-bottom, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/19040b6538a6bdeed7c833cfb44614b6/cf1/744bcdff675143b3b9b15d9f93d2e618.93c2cc329b1fe1893cfbc28c56960522->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf1/744bcdff675143b3b9b15d9f93d2e618-bottom] into tmpdir=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/19040b6538a6bdeed7c833cfb44614b6/.tmp, totalSize=19.9 K 2023-05-10 04:53:46,129 INFO [RS:2;jenkins-hbase4:37823-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/19040b6538a6bdeed7c833cfb44614b6/cf2/b1cbcf3117244e049435fa8e701a7a80.93c2cc329b1fe1893cfbc28c56960522->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf2/b1cbcf3117244e049435fa8e701a7a80-bottom, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/19040b6538a6bdeed7c833cfb44614b6/cf2/bd68bbeaf0094fd0a415560924da2bce.93c2cc329b1fe1893cfbc28c56960522->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf2/bd68bbeaf0094fd0a415560924da2bce-bottom, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/19040b6538a6bdeed7c833cfb44614b6/cf2/76f6f9416941403596b6ff3caccb861c.93c2cc329b1fe1893cfbc28c56960522->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf2/76f6f9416941403596b6ff3caccb861c-bottom, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/19040b6538a6bdeed7c833cfb44614b6/cf2/4ee8175dfe724214ad50b9f4f420f989.93c2cc329b1fe1893cfbc28c56960522->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf2/4ee8175dfe724214ad50b9f4f420f989-bottom] into tmpdir=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/19040b6538a6bdeed7c833cfb44614b6/.tmp, totalSize=19.9 K 2023-05-10 04:53:46,129 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionUnevenDaughter,,1683694424955.19040b6538a6bdeed7c833cfb44614b6.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1683694426129"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1683694426129"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1683694426129"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683694426129"}]},"ts":"1683694426129"} 2023-05-10 04:53:46,130 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-05-10 04:53:46,131 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionUnevenDaughter 054f69c06f126aad97b290410d5d6188 2023-05-10 04:53:46,131 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionUnevenDaughter,26,1683694424955.054f69c06f126aad97b290410d5d6188.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-10 04:53:46,131 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 054f69c06f126aad97b290410d5d6188 2023-05-10 04:53:46,131 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 054f69c06f126aad97b290410d5d6188 2023-05-10 04:53:46,132 DEBUG [RS:2;jenkins-hbase4:37823-shortCompactions-0] compactions.Compactor(207): Compacting 9333fac35a6e43f2ae1eb05c2a8c43e2.93c2cc329b1fe1893cfbc28c56960522, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1683694422892 2023-05-10 04:53:46,132 DEBUG [RS:2;jenkins-hbase4:37823-longCompactions-0] compactions.Compactor(207): Compacting b1cbcf3117244e049435fa8e701a7a80.93c2cc329b1fe1893cfbc28c56960522, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1683694422892 2023-05-10 04:53:46,134 DEBUG [RS:2;jenkins-hbase4:37823-longCompactions-0] compactions.Compactor(207): Compacting bd68bbeaf0094fd0a415560924da2bce.93c2cc329b1fe1893cfbc28c56960522, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1683694423689 2023-05-10 04:53:46,134 DEBUG [RS:2;jenkins-hbase4:37823-shortCompactions-0] compactions.Compactor(207): Compacting 6fc2425571254b1b8437fabc5c780c71.93c2cc329b1fe1893cfbc28c56960522, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1683694423689 2023-05-10 04:53:46,134 INFO [StoreOpener-054f69c06f126aad97b290410d5d6188-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 054f69c06f126aad97b290410d5d6188 2023-05-10 04:53:46,134 DEBUG [RS:2;jenkins-hbase4:37823-longCompactions-0] compactions.Compactor(207): Compacting 76f6f9416941403596b6ff3caccb861c.93c2cc329b1fe1893cfbc28c56960522, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1683694423948 2023-05-10 04:53:46,135 DEBUG [RS:2;jenkins-hbase4:37823-longCompactions-0] compactions.Compactor(207): Compacting 4ee8175dfe724214ad50b9f4f420f989.93c2cc329b1fe1893cfbc28c56960522, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1683694424194 2023-05-10 04:53:46,135 DEBUG [RS:2;jenkins-hbase4:37823-shortCompactions-0] compactions.Compactor(207): Compacting be24ca36992745b7a1a3b25d9bc68d53.93c2cc329b1fe1893cfbc28c56960522, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1683694423948 2023-05-10 04:53:46,136 DEBUG [StoreOpener-054f69c06f126aad97b290410d5d6188-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/054f69c06f126aad97b290410d5d6188/cf1 2023-05-10 04:53:46,136 DEBUG [StoreOpener-054f69c06f126aad97b290410d5d6188-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/054f69c06f126aad97b290410d5d6188/cf1 2023-05-10 04:53:46,137 INFO [StoreOpener-054f69c06f126aad97b290410d5d6188-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 054f69c06f126aad97b290410d5d6188 columnFamilyName cf1 2023-05-10 04:53:46,137 DEBUG [RS:2;jenkins-hbase4:37823-shortCompactions-0] compactions.Compactor(207): Compacting 744bcdff675143b3b9b15d9f93d2e618.93c2cc329b1fe1893cfbc28c56960522, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1683694424194 2023-05-10 04:53:46,138 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=18, resume processing ppid=15 2023-05-10 04:53:46,139 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=18, ppid=15, state=SUCCESS; OpenRegionProcedure 19040b6538a6bdeed7c833cfb44614b6, server=jenkins-hbase4.apache.org,37823,1683694419534 in 277 msec 2023-05-10 04:53:46,143 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=15, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=19040b6538a6bdeed7c833cfb44614b6, ASSIGN in 444 msec 2023-05-10 04:53:46,159 INFO [RS:2;jenkins-hbase4:37823-longCompactions-0] throttle.PressureAwareThroughputController(145): 19040b6538a6bdeed7c833cfb44614b6#cf2#compaction#14 average throughput is 0.43 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-05-10 04:53:46,159 INFO [RS:2;jenkins-hbase4:37823-shortCompactions-0] throttle.PressureAwareThroughputController(145): 19040b6538a6bdeed7c833cfb44614b6#cf1#compaction#15 average throughput is 0.21 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-05-10 04:53:46,166 DEBUG [StoreOpener-054f69c06f126aad97b290410d5d6188-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/054f69c06f126aad97b290410d5d6188/cf1/6fc2425571254b1b8437fabc5c780c71.93c2cc329b1fe1893cfbc28c56960522->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf1/6fc2425571254b1b8437fabc5c780c71-top 2023-05-10 04:53:46,172 DEBUG [StoreOpener-054f69c06f126aad97b290410d5d6188-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/054f69c06f126aad97b290410d5d6188/cf1/744bcdff675143b3b9b15d9f93d2e618.93c2cc329b1fe1893cfbc28c56960522->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf1/744bcdff675143b3b9b15d9f93d2e618-top 2023-05-10 04:53:46,178 DEBUG [StoreOpener-054f69c06f126aad97b290410d5d6188-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/054f69c06f126aad97b290410d5d6188/cf1/9333fac35a6e43f2ae1eb05c2a8c43e2.93c2cc329b1fe1893cfbc28c56960522->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf1/9333fac35a6e43f2ae1eb05c2a8c43e2-top 2023-05-10 04:53:46,190 DEBUG [StoreOpener-054f69c06f126aad97b290410d5d6188-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/054f69c06f126aad97b290410d5d6188/cf1/be24ca36992745b7a1a3b25d9bc68d53.93c2cc329b1fe1893cfbc28c56960522->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf1/be24ca36992745b7a1a3b25d9bc68d53-top 2023-05-10 04:53:46,199 DEBUG [StoreOpener-054f69c06f126aad97b290410d5d6188-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/054f69c06f126aad97b290410d5d6188/cf1/testSplitTableRegionUnevenDaughter=93c2cc329b1fe1893cfbc28c56960522-427cc49d3a8e4682b202f4b0673d8806 2023-05-10 04:53:46,204 DEBUG [StoreOpener-054f69c06f126aad97b290410d5d6188-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/054f69c06f126aad97b290410d5d6188/cf1/testSplitTableRegionUnevenDaughter=93c2cc329b1fe1893cfbc28c56960522-a9634e7ee1844806a05a96cd8e89dd4d 2023-05-10 04:53:46,205 DEBUG [RS:2;jenkins-hbase4:37823-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/19040b6538a6bdeed7c833cfb44614b6/.tmp/cf1/b9b226020897466e9944b392670962f2 as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/19040b6538a6bdeed7c833cfb44614b6/cf1/b9b226020897466e9944b392670962f2 2023-05-10 04:53:46,206 DEBUG [RS:2;jenkins-hbase4:37823-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/19040b6538a6bdeed7c833cfb44614b6/.tmp/cf2/d185e4b0cdc3444ea290b89728d398a0 as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/19040b6538a6bdeed7c833cfb44614b6/cf2/d185e4b0cdc3444ea290b89728d398a0 2023-05-10 04:53:46,211 DEBUG [StoreOpener-054f69c06f126aad97b290410d5d6188-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/054f69c06f126aad97b290410d5d6188/cf1/testSplitTableRegionUnevenDaughter=93c2cc329b1fe1893cfbc28c56960522-e4b432d539b1457f9648cdf383c363c9 2023-05-10 04:53:46,212 INFO [StoreOpener-054f69c06f126aad97b290410d5d6188-1] regionserver.HStore(310): Store=054f69c06f126aad97b290410d5d6188/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-10 04:53:46,213 INFO [StoreOpener-054f69c06f126aad97b290410d5d6188-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 054f69c06f126aad97b290410d5d6188 2023-05-10 04:53:46,215 DEBUG [StoreOpener-054f69c06f126aad97b290410d5d6188-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/054f69c06f126aad97b290410d5d6188/cf2 2023-05-10 04:53:46,215 DEBUG [StoreOpener-054f69c06f126aad97b290410d5d6188-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/054f69c06f126aad97b290410d5d6188/cf2 2023-05-10 04:53:46,215 INFO [StoreOpener-054f69c06f126aad97b290410d5d6188-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 054f69c06f126aad97b290410d5d6188 columnFamilyName cf2 2023-05-10 04:53:46,226 INFO [RS:2;jenkins-hbase4:37823-longCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 19040b6538a6bdeed7c833cfb44614b6/cf2 of 19040b6538a6bdeed7c833cfb44614b6 into d185e4b0cdc3444ea290b89728d398a0(size=5.5 K), total size for store is 5.5 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-05-10 04:53:46,226 INFO [RS:2;jenkins-hbase4:37823-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 19040b6538a6bdeed7c833cfb44614b6/cf1 of 19040b6538a6bdeed7c833cfb44614b6 into b9b226020897466e9944b392670962f2(size=5.5 K), total size for store is 5.5 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-05-10 04:53:46,228 DEBUG [RS:2;jenkins-hbase4:37823-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 19040b6538a6bdeed7c833cfb44614b6: 2023-05-10 04:53:46,228 DEBUG [RS:2;jenkins-hbase4:37823-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 19040b6538a6bdeed7c833cfb44614b6: 2023-05-10 04:53:46,228 INFO [RS:2;jenkins-hbase4:37823-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,,1683694424955.19040b6538a6bdeed7c833cfb44614b6., storeName=19040b6538a6bdeed7c833cfb44614b6/cf1, priority=12, startTime=1683694426124; duration=0sec 2023-05-10 04:53:46,228 INFO [RS:2;jenkins-hbase4:37823-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,,1683694424955.19040b6538a6bdeed7c833cfb44614b6., storeName=19040b6538a6bdeed7c833cfb44614b6/cf2, priority=12, startTime=1683694426125; duration=0sec 2023-05-10 04:53:46,228 DEBUG [RS:2;jenkins-hbase4:37823-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-10 04:53:46,229 DEBUG [RS:2;jenkins-hbase4:37823-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-10 04:53:46,233 DEBUG [StoreOpener-054f69c06f126aad97b290410d5d6188-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/054f69c06f126aad97b290410d5d6188/cf2/4ee8175dfe724214ad50b9f4f420f989.93c2cc329b1fe1893cfbc28c56960522->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf2/4ee8175dfe724214ad50b9f4f420f989-top 2023-05-10 04:53:46,238 DEBUG [StoreOpener-054f69c06f126aad97b290410d5d6188-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/054f69c06f126aad97b290410d5d6188/cf2/76f6f9416941403596b6ff3caccb861c.93c2cc329b1fe1893cfbc28c56960522->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf2/76f6f9416941403596b6ff3caccb861c-top 2023-05-10 04:53:46,244 DEBUG [StoreOpener-054f69c06f126aad97b290410d5d6188-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/054f69c06f126aad97b290410d5d6188/cf2/b1cbcf3117244e049435fa8e701a7a80.93c2cc329b1fe1893cfbc28c56960522->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf2/b1cbcf3117244e049435fa8e701a7a80-top 2023-05-10 04:53:46,249 DEBUG [StoreOpener-054f69c06f126aad97b290410d5d6188-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/054f69c06f126aad97b290410d5d6188/cf2/bd68bbeaf0094fd0a415560924da2bce.93c2cc329b1fe1893cfbc28c56960522->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf2/bd68bbeaf0094fd0a415560924da2bce-top 2023-05-10 04:53:46,255 DEBUG [StoreOpener-054f69c06f126aad97b290410d5d6188-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/054f69c06f126aad97b290410d5d6188/cf2/testSplitTableRegionUnevenDaughter=93c2cc329b1fe1893cfbc28c56960522-48782fe6bd7d4d8996815ff75cdfac23 2023-05-10 04:53:46,259 DEBUG [StoreOpener-054f69c06f126aad97b290410d5d6188-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/054f69c06f126aad97b290410d5d6188/cf2/testSplitTableRegionUnevenDaughter=93c2cc329b1fe1893cfbc28c56960522-7a78adce76124917861cfd93394e25fd 2023-05-10 04:53:46,264 DEBUG [StoreOpener-054f69c06f126aad97b290410d5d6188-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/054f69c06f126aad97b290410d5d6188/cf2/testSplitTableRegionUnevenDaughter=93c2cc329b1fe1893cfbc28c56960522-a4dcfc1811054c03bee550c410a35fbb 2023-05-10 04:53:46,264 INFO [StoreOpener-054f69c06f126aad97b290410d5d6188-1] regionserver.HStore(310): Store=054f69c06f126aad97b290410d5d6188/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-10 04:53:46,265 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/054f69c06f126aad97b290410d5d6188 2023-05-10 04:53:46,267 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/054f69c06f126aad97b290410d5d6188 2023-05-10 04:53:46,269 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionUnevenDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-05-10 04:53:46,271 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 054f69c06f126aad97b290410d5d6188 2023-05-10 04:53:46,271 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 054f69c06f126aad97b290410d5d6188; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66609869, jitterRate=-0.007435604929924011}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-05-10 04:53:46,272 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 054f69c06f126aad97b290410d5d6188: 2023-05-10 04:53:46,273 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2334): Post open deploy tasks for testSplitTableRegionUnevenDaughter,26,1683694424955.054f69c06f126aad97b290410d5d6188., pid=17, masterSystemTime=1683694426003 2023-05-10 04:53:46,273 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-10 04:53:46,274 DEBUG [RS:2;jenkins-hbase4:37823-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-05-10 04:53:46,274 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-05-10 04:53:46,274 DEBUG [RS:2;jenkins-hbase4:37823-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-05-10 04:53:46,274 INFO [RS:2;jenkins-hbase4:37823-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,26,1683694424955.054f69c06f126aad97b290410d5d6188. 2023-05-10 04:53:46,274 DEBUG [RS:2;jenkins-hbase4:37823-shortCompactions-0] regionserver.HStore(1912): 054f69c06f126aad97b290410d5d6188/cf1 is initiating minor compaction (all files) 2023-05-10 04:53:46,274 INFO [RS:2;jenkins-hbase4:37823-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,26,1683694424955.054f69c06f126aad97b290410d5d6188. 2023-05-10 04:53:46,274 DEBUG [RS:2;jenkins-hbase4:37823-longCompactions-0] regionserver.HStore(1912): 054f69c06f126aad97b290410d5d6188/cf2 is initiating minor compaction (all files) 2023-05-10 04:53:46,274 INFO [RS:2;jenkins-hbase4:37823-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 054f69c06f126aad97b290410d5d6188/cf1 in testSplitTableRegionUnevenDaughter,26,1683694424955.054f69c06f126aad97b290410d5d6188. 2023-05-10 04:53:46,275 INFO [RS:2;jenkins-hbase4:37823-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 054f69c06f126aad97b290410d5d6188/cf2 in testSplitTableRegionUnevenDaughter,26,1683694424955.054f69c06f126aad97b290410d5d6188. 2023-05-10 04:53:46,275 INFO [RS:2;jenkins-hbase4:37823-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/054f69c06f126aad97b290410d5d6188/cf1/9333fac35a6e43f2ae1eb05c2a8c43e2.93c2cc329b1fe1893cfbc28c56960522->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf1/9333fac35a6e43f2ae1eb05c2a8c43e2-top, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/054f69c06f126aad97b290410d5d6188/cf1/6fc2425571254b1b8437fabc5c780c71.93c2cc329b1fe1893cfbc28c56960522->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf1/6fc2425571254b1b8437fabc5c780c71-top, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/054f69c06f126aad97b290410d5d6188/cf1/be24ca36992745b7a1a3b25d9bc68d53.93c2cc329b1fe1893cfbc28c56960522->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf1/be24ca36992745b7a1a3b25d9bc68d53-top, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/054f69c06f126aad97b290410d5d6188/cf1/744bcdff675143b3b9b15d9f93d2e618.93c2cc329b1fe1893cfbc28c56960522->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf1/744bcdff675143b3b9b15d9f93d2e618-top, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/054f69c06f126aad97b290410d5d6188/cf1/testSplitTableRegionUnevenDaughter=93c2cc329b1fe1893cfbc28c56960522-e4b432d539b1457f9648cdf383c363c9, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/054f69c06f126aad97b290410d5d6188/cf1/testSplitTableRegionUnevenDaughter=93c2cc329b1fe1893cfbc28c56960522-a9634e7ee1844806a05a96cd8e89dd4d, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/054f69c06f126aad97b290410d5d6188/cf1/testSplitTableRegionUnevenDaughter=93c2cc329b1fe1893cfbc28c56960522-427cc49d3a8e4682b202f4b0673d8806] into tmpdir=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/054f69c06f126aad97b290410d5d6188/.tmp, totalSize=34.9 K 2023-05-10 04:53:46,275 INFO [RS:2;jenkins-hbase4:37823-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/054f69c06f126aad97b290410d5d6188/cf2/b1cbcf3117244e049435fa8e701a7a80.93c2cc329b1fe1893cfbc28c56960522->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf2/b1cbcf3117244e049435fa8e701a7a80-top, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/054f69c06f126aad97b290410d5d6188/cf2/bd68bbeaf0094fd0a415560924da2bce.93c2cc329b1fe1893cfbc28c56960522->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf2/bd68bbeaf0094fd0a415560924da2bce-top, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/054f69c06f126aad97b290410d5d6188/cf2/76f6f9416941403596b6ff3caccb861c.93c2cc329b1fe1893cfbc28c56960522->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf2/76f6f9416941403596b6ff3caccb861c-top, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/054f69c06f126aad97b290410d5d6188/cf2/4ee8175dfe724214ad50b9f4f420f989.93c2cc329b1fe1893cfbc28c56960522->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf2/4ee8175dfe724214ad50b9f4f420f989-top, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/054f69c06f126aad97b290410d5d6188/cf2/testSplitTableRegionUnevenDaughter=93c2cc329b1fe1893cfbc28c56960522-a4dcfc1811054c03bee550c410a35fbb, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/054f69c06f126aad97b290410d5d6188/cf2/testSplitTableRegionUnevenDaughter=93c2cc329b1fe1893cfbc28c56960522-7a78adce76124917861cfd93394e25fd, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/054f69c06f126aad97b290410d5d6188/cf2/testSplitTableRegionUnevenDaughter=93c2cc329b1fe1893cfbc28c56960522-48782fe6bd7d4d8996815ff75cdfac23] into tmpdir=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/054f69c06f126aad97b290410d5d6188/.tmp, totalSize=34.9 K 2023-05-10 04:53:46,276 DEBUG [RS:2;jenkins-hbase4:37823-shortCompactions-0] compactions.Compactor(207): Compacting 9333fac35a6e43f2ae1eb05c2a8c43e2.93c2cc329b1fe1893cfbc28c56960522, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1683694422892 2023-05-10 04:53:46,278 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2361): Finished post open deploy task for testSplitTableRegionUnevenDaughter,26,1683694424955.054f69c06f126aad97b290410d5d6188. 2023-05-10 04:53:46,279 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionUnevenDaughter,26,1683694424955.054f69c06f126aad97b290410d5d6188. 2023-05-10 04:53:46,279 DEBUG [RS:2;jenkins-hbase4:37823-longCompactions-0] compactions.Compactor(207): Compacting b1cbcf3117244e049435fa8e701a7a80.93c2cc329b1fe1893cfbc28c56960522, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1683694422892 2023-05-10 04:53:46,279 DEBUG [RS:2;jenkins-hbase4:37823-shortCompactions-0] compactions.Compactor(207): Compacting 6fc2425571254b1b8437fabc5c780c71.93c2cc329b1fe1893cfbc28c56960522, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1683694423689 2023-05-10 04:53:46,280 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=16 updating hbase:meta row=054f69c06f126aad97b290410d5d6188, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:46,280 DEBUG [RS:2;jenkins-hbase4:37823-longCompactions-0] compactions.Compactor(207): Compacting bd68bbeaf0094fd0a415560924da2bce.93c2cc329b1fe1893cfbc28c56960522, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1683694423689 2023-05-10 04:53:46,280 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionUnevenDaughter,26,1683694424955.054f69c06f126aad97b290410d5d6188.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1683694426280"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1683694426280"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1683694426280"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683694426280"}]},"ts":"1683694426280"} 2023-05-10 04:53:46,280 DEBUG [RS:2;jenkins-hbase4:37823-shortCompactions-0] compactions.Compactor(207): Compacting be24ca36992745b7a1a3b25d9bc68d53.93c2cc329b1fe1893cfbc28c56960522, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1683694423948 2023-05-10 04:53:46,281 DEBUG [RS:2;jenkins-hbase4:37823-longCompactions-0] compactions.Compactor(207): Compacting 76f6f9416941403596b6ff3caccb861c.93c2cc329b1fe1893cfbc28c56960522, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1683694423948 2023-05-10 04:53:46,281 DEBUG [RS:2;jenkins-hbase4:37823-shortCompactions-0] compactions.Compactor(207): Compacting 744bcdff675143b3b9b15d9f93d2e618.93c2cc329b1fe1893cfbc28c56960522, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1683694424194 2023-05-10 04:53:46,283 DEBUG [RS:2;jenkins-hbase4:37823-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=93c2cc329b1fe1893cfbc28c56960522-e4b432d539b1457f9648cdf383c363c9, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1683694424460 2023-05-10 04:53:46,283 DEBUG [RS:2;jenkins-hbase4:37823-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=93c2cc329b1fe1893cfbc28c56960522-a9634e7ee1844806a05a96cd8e89dd4d, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1683694424700 2023-05-10 04:53:46,283 DEBUG [RS:2;jenkins-hbase4:37823-longCompactions-0] compactions.Compactor(207): Compacting 4ee8175dfe724214ad50b9f4f420f989.93c2cc329b1fe1893cfbc28c56960522, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1683694424194 2023-05-10 04:53:46,284 DEBUG [RS:2;jenkins-hbase4:37823-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=93c2cc329b1fe1893cfbc28c56960522-427cc49d3a8e4682b202f4b0673d8806, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1683694424930 2023-05-10 04:53:46,284 DEBUG [RS:2;jenkins-hbase4:37823-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=93c2cc329b1fe1893cfbc28c56960522-a4dcfc1811054c03bee550c410a35fbb, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1683694424460 2023-05-10 04:53:46,285 DEBUG [RS:2;jenkins-hbase4:37823-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=93c2cc329b1fe1893cfbc28c56960522-7a78adce76124917861cfd93394e25fd, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1683694424700 2023-05-10 04:53:46,285 DEBUG [RS:2;jenkins-hbase4:37823-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=93c2cc329b1fe1893cfbc28c56960522-48782fe6bd7d4d8996815ff75cdfac23, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1683694424930 2023-05-10 04:53:46,288 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=17, resume processing ppid=16 2023-05-10 04:53:46,288 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=17, ppid=16, state=SUCCESS; OpenRegionProcedure 054f69c06f126aad97b290410d5d6188, server=jenkins-hbase4.apache.org,37823,1683694419534 in 433 msec 2023-05-10 04:53:46,292 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=16, resume processing ppid=12 2023-05-10 04:53:46,292 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=16, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=054f69c06f126aad97b290410d5d6188, ASSIGN in 593 msec 2023-05-10 04:53:46,295 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=93c2cc329b1fe1893cfbc28c56960522, daughterA=19040b6538a6bdeed7c833cfb44614b6, daughterB=054f69c06f126aad97b290410d5d6188 in 1.3370 sec 2023-05-10 04:53:46,313 INFO [RS:2;jenkins-hbase4:37823-longCompactions-0] throttle.PressureAwareThroughputController(145): 054f69c06f126aad97b290410d5d6188#cf2#compaction#16 average throughput is 0.64 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-05-10 04:53:46,317 INFO [RS:2;jenkins-hbase4:37823-shortCompactions-0] throttle.PressureAwareThroughputController(145): 054f69c06f126aad97b290410d5d6188#cf1#compaction#17 average throughput is 0.64 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-05-10 04:53:46,362 DEBUG [RS:2;jenkins-hbase4:37823-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/054f69c06f126aad97b290410d5d6188/.tmp/cf2/a3e2994715ee4e608e3cf09a744ae646 as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/054f69c06f126aad97b290410d5d6188/cf2/a3e2994715ee4e608e3cf09a744ae646 2023-05-10 04:53:46,363 DEBUG [RS:2;jenkins-hbase4:37823-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/054f69c06f126aad97b290410d5d6188/.tmp/cf1/564780056d5d4ccf975765c3601c012c as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/054f69c06f126aad97b290410d5d6188/cf1/564780056d5d4ccf975765c3601c012c 2023-05-10 04:53:46,371 INFO [RS:2;jenkins-hbase4:37823-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 054f69c06f126aad97b290410d5d6188/cf2 of 054f69c06f126aad97b290410d5d6188 into a3e2994715ee4e608e3cf09a744ae646(size=6.7 K), total size for store is 6.7 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-05-10 04:53:46,371 INFO [RS:2;jenkins-hbase4:37823-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 054f69c06f126aad97b290410d5d6188/cf1 of 054f69c06f126aad97b290410d5d6188 into 564780056d5d4ccf975765c3601c012c(size=6.7 K), total size for store is 6.7 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-05-10 04:53:46,372 DEBUG [RS:2;jenkins-hbase4:37823-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 054f69c06f126aad97b290410d5d6188: 2023-05-10 04:53:46,372 DEBUG [RS:2;jenkins-hbase4:37823-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 054f69c06f126aad97b290410d5d6188: 2023-05-10 04:53:46,372 INFO [RS:2;jenkins-hbase4:37823-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,26,1683694424955.054f69c06f126aad97b290410d5d6188., storeName=054f69c06f126aad97b290410d5d6188/cf2, priority=9, startTime=1683694426274; duration=0sec 2023-05-10 04:53:46,373 INFO [RS:2;jenkins-hbase4:37823-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,26,1683694424955.054f69c06f126aad97b290410d5d6188., storeName=054f69c06f126aad97b290410d5d6188/cf1, priority=9, startTime=1683694426273; duration=0sec 2023-05-10 04:53:46,373 DEBUG [RS:2;jenkins-hbase4:37823-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-10 04:53:46,373 DEBUG [RS:2;jenkins-hbase4:37823-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-10 04:53:46,464 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37823] ipc.CallRunner(144): callId: 106 service: ClientService methodName: Scan size: 157 connection: 172.31.14.131:55416 deadline: 1683694486463, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitTableRegionUnevenDaughter,,1683694422380.93c2cc329b1fe1893cfbc28c56960522. is not online on jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:46,601 DEBUG [Listener at localhost/45007] client.ClientScanner(240): Advancing internal scanner to startKey at '26', inclusive 2023-05-10 04:53:46,612 INFO [Listener at localhost/45007] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-05-10 04:53:46,615 DEBUG [Listener at localhost/45007] client.ClientScanner(240): Advancing internal scanner to startKey at '26', inclusive 2023-05-10 04:53:46,646 WARN [Listener at localhost/45007] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-05-10 04:53:46,656 INFO [Listener at localhost/45007] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionUnevenDaughter 2023-05-10 04:53:46,661 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testSplitTableRegionUnevenDaughter 2023-05-10 04:53:46,668 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] procedure2.ProcedureExecutor(1029): Stored pid=19, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:53:46,673 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683694426673"}]},"ts":"1683694426673"} 2023-05-10 04:53:46,675 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=DISABLING in hbase:meta 2023-05-10 04:53:46,677 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionUnevenDaughter to state=DISABLING 2023-05-10 04:53:46,679 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-05-10 04:53:46,679 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=19040b6538a6bdeed7c833cfb44614b6, UNASSIGN}, {pid=21, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=054f69c06f126aad97b290410d5d6188, UNASSIGN}] 2023-05-10 04:53:46,680 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=21, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=054f69c06f126aad97b290410d5d6188, UNASSIGN 2023-05-10 04:53:46,681 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=21 updating hbase:meta row=054f69c06f126aad97b290410d5d6188, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:46,681 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,26,1683694424955.054f69c06f126aad97b290410d5d6188.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1683694426681"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683694426681"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683694426681"}]},"ts":"1683694426681"} 2023-05-10 04:53:46,683 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=22, ppid=21, state=RUNNABLE; CloseRegionProcedure 054f69c06f126aad97b290410d5d6188, server=jenkins-hbase4.apache.org,37823,1683694419534}] 2023-05-10 04:53:46,686 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=19040b6538a6bdeed7c833cfb44614b6, UNASSIGN 2023-05-10 04:53:46,687 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=19040b6538a6bdeed7c833cfb44614b6, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:46,688 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1683694424955.19040b6538a6bdeed7c833cfb44614b6.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1683694426687"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683694426687"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683694426687"}]},"ts":"1683694426687"} 2023-05-10 04:53:46,689 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=23, ppid=20, state=RUNNABLE; CloseRegionProcedure 19040b6538a6bdeed7c833cfb44614b6, server=jenkins-hbase4.apache.org,37823,1683694419534}] 2023-05-10 04:53:46,783 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-05-10 04:53:46,836 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 19040b6538a6bdeed7c833cfb44614b6 2023-05-10 04:53:46,836 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 19040b6538a6bdeed7c833cfb44614b6, disabling compactions & flushes 2023-05-10 04:53:46,836 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionUnevenDaughter,,1683694424955.19040b6538a6bdeed7c833cfb44614b6. 2023-05-10 04:53:46,836 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,,1683694424955.19040b6538a6bdeed7c833cfb44614b6. 2023-05-10 04:53:46,836 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionUnevenDaughter,,1683694424955.19040b6538a6bdeed7c833cfb44614b6. after waiting 0 ms 2023-05-10 04:53:46,836 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionUnevenDaughter,,1683694424955.19040b6538a6bdeed7c833cfb44614b6. 2023-05-10 04:53:46,842 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1683694424955.19040b6538a6bdeed7c833cfb44614b6.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/19040b6538a6bdeed7c833cfb44614b6/cf1/9333fac35a6e43f2ae1eb05c2a8c43e2.93c2cc329b1fe1893cfbc28c56960522->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf1/9333fac35a6e43f2ae1eb05c2a8c43e2-bottom, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/19040b6538a6bdeed7c833cfb44614b6/cf1/6fc2425571254b1b8437fabc5c780c71.93c2cc329b1fe1893cfbc28c56960522->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf1/6fc2425571254b1b8437fabc5c780c71-bottom, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/19040b6538a6bdeed7c833cfb44614b6/cf1/be24ca36992745b7a1a3b25d9bc68d53.93c2cc329b1fe1893cfbc28c56960522->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf1/be24ca36992745b7a1a3b25d9bc68d53-bottom, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/19040b6538a6bdeed7c833cfb44614b6/cf1/744bcdff675143b3b9b15d9f93d2e618.93c2cc329b1fe1893cfbc28c56960522->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf1/744bcdff675143b3b9b15d9f93d2e618-bottom] to archive 2023-05-10 04:53:46,844 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1683694424955.19040b6538a6bdeed7c833cfb44614b6.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-05-10 04:53:46,844 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-05-10 04:53:46,855 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1683694424955.19040b6538a6bdeed7c833cfb44614b6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/19040b6538a6bdeed7c833cfb44614b6/cf1/9333fac35a6e43f2ae1eb05c2a8c43e2.93c2cc329b1fe1893cfbc28c56960522 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionUnevenDaughter/19040b6538a6bdeed7c833cfb44614b6/cf1/9333fac35a6e43f2ae1eb05c2a8c43e2.93c2cc329b1fe1893cfbc28c56960522 2023-05-10 04:53:46,860 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1683694424955.19040b6538a6bdeed7c833cfb44614b6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/19040b6538a6bdeed7c833cfb44614b6/cf1/6fc2425571254b1b8437fabc5c780c71.93c2cc329b1fe1893cfbc28c56960522 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionUnevenDaughter/19040b6538a6bdeed7c833cfb44614b6/cf1/6fc2425571254b1b8437fabc5c780c71.93c2cc329b1fe1893cfbc28c56960522 2023-05-10 04:53:46,862 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1683694424955.19040b6538a6bdeed7c833cfb44614b6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/19040b6538a6bdeed7c833cfb44614b6/cf1/be24ca36992745b7a1a3b25d9bc68d53.93c2cc329b1fe1893cfbc28c56960522 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionUnevenDaughter/19040b6538a6bdeed7c833cfb44614b6/cf1/be24ca36992745b7a1a3b25d9bc68d53.93c2cc329b1fe1893cfbc28c56960522 2023-05-10 04:53:46,864 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1683694424955.19040b6538a6bdeed7c833cfb44614b6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/19040b6538a6bdeed7c833cfb44614b6/cf1/744bcdff675143b3b9b15d9f93d2e618.93c2cc329b1fe1893cfbc28c56960522 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionUnevenDaughter/19040b6538a6bdeed7c833cfb44614b6/cf1/744bcdff675143b3b9b15d9f93d2e618.93c2cc329b1fe1893cfbc28c56960522 2023-05-10 04:53:46,899 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1683694424955.19040b6538a6bdeed7c833cfb44614b6.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/19040b6538a6bdeed7c833cfb44614b6/cf2/b1cbcf3117244e049435fa8e701a7a80.93c2cc329b1fe1893cfbc28c56960522->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf2/b1cbcf3117244e049435fa8e701a7a80-bottom, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/19040b6538a6bdeed7c833cfb44614b6/cf2/bd68bbeaf0094fd0a415560924da2bce.93c2cc329b1fe1893cfbc28c56960522->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf2/bd68bbeaf0094fd0a415560924da2bce-bottom, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/19040b6538a6bdeed7c833cfb44614b6/cf2/76f6f9416941403596b6ff3caccb861c.93c2cc329b1fe1893cfbc28c56960522->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf2/76f6f9416941403596b6ff3caccb861c-bottom, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/19040b6538a6bdeed7c833cfb44614b6/cf2/4ee8175dfe724214ad50b9f4f420f989.93c2cc329b1fe1893cfbc28c56960522->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf2/4ee8175dfe724214ad50b9f4f420f989-bottom] to archive 2023-05-10 04:53:46,900 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1683694424955.19040b6538a6bdeed7c833cfb44614b6.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-05-10 04:53:46,904 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1683694424955.19040b6538a6bdeed7c833cfb44614b6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/19040b6538a6bdeed7c833cfb44614b6/cf2/b1cbcf3117244e049435fa8e701a7a80.93c2cc329b1fe1893cfbc28c56960522 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionUnevenDaughter/19040b6538a6bdeed7c833cfb44614b6/cf2/b1cbcf3117244e049435fa8e701a7a80.93c2cc329b1fe1893cfbc28c56960522 2023-05-10 04:53:46,910 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1683694424955.19040b6538a6bdeed7c833cfb44614b6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/19040b6538a6bdeed7c833cfb44614b6/cf2/bd68bbeaf0094fd0a415560924da2bce.93c2cc329b1fe1893cfbc28c56960522 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionUnevenDaughter/19040b6538a6bdeed7c833cfb44614b6/cf2/bd68bbeaf0094fd0a415560924da2bce.93c2cc329b1fe1893cfbc28c56960522 2023-05-10 04:53:46,912 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1683694424955.19040b6538a6bdeed7c833cfb44614b6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/19040b6538a6bdeed7c833cfb44614b6/cf2/76f6f9416941403596b6ff3caccb861c.93c2cc329b1fe1893cfbc28c56960522 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionUnevenDaughter/19040b6538a6bdeed7c833cfb44614b6/cf2/76f6f9416941403596b6ff3caccb861c.93c2cc329b1fe1893cfbc28c56960522 2023-05-10 04:53:46,914 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1683694424955.19040b6538a6bdeed7c833cfb44614b6.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/19040b6538a6bdeed7c833cfb44614b6/cf2/4ee8175dfe724214ad50b9f4f420f989.93c2cc329b1fe1893cfbc28c56960522 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionUnevenDaughter/19040b6538a6bdeed7c833cfb44614b6/cf2/4ee8175dfe724214ad50b9f4f420f989.93c2cc329b1fe1893cfbc28c56960522 2023-05-10 04:53:46,931 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/19040b6538a6bdeed7c833cfb44614b6/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-05-10 04:53:46,931 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-10 04:53:46,932 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionUnevenDaughter,,1683694424955.19040b6538a6bdeed7c833cfb44614b6. 2023-05-10 04:53:46,932 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 19040b6538a6bdeed7c833cfb44614b6: 2023-05-10 04:53:46,936 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 19040b6538a6bdeed7c833cfb44614b6 2023-05-10 04:53:46,936 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 054f69c06f126aad97b290410d5d6188 2023-05-10 04:53:46,937 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 054f69c06f126aad97b290410d5d6188, disabling compactions & flushes 2023-05-10 04:53:46,937 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionUnevenDaughter,26,1683694424955.054f69c06f126aad97b290410d5d6188. 2023-05-10 04:53:46,937 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,26,1683694424955.054f69c06f126aad97b290410d5d6188. 2023-05-10 04:53:46,937 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionUnevenDaughter,26,1683694424955.054f69c06f126aad97b290410d5d6188. after waiting 0 ms 2023-05-10 04:53:46,937 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionUnevenDaughter,26,1683694424955.054f69c06f126aad97b290410d5d6188. 2023-05-10 04:53:46,937 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=19040b6538a6bdeed7c833cfb44614b6, regionState=CLOSED 2023-05-10 04:53:46,938 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,,1683694424955.19040b6538a6bdeed7c833cfb44614b6.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1683694426937"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683694426937"}]},"ts":"1683694426937"} 2023-05-10 04:53:46,940 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1683694424955.054f69c06f126aad97b290410d5d6188.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/054f69c06f126aad97b290410d5d6188/cf1/9333fac35a6e43f2ae1eb05c2a8c43e2.93c2cc329b1fe1893cfbc28c56960522->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf1/9333fac35a6e43f2ae1eb05c2a8c43e2-top, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/054f69c06f126aad97b290410d5d6188/cf1/6fc2425571254b1b8437fabc5c780c71.93c2cc329b1fe1893cfbc28c56960522->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf1/6fc2425571254b1b8437fabc5c780c71-top, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/054f69c06f126aad97b290410d5d6188/cf1/be24ca36992745b7a1a3b25d9bc68d53.93c2cc329b1fe1893cfbc28c56960522->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf1/be24ca36992745b7a1a3b25d9bc68d53-top, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/054f69c06f126aad97b290410d5d6188/cf1/744bcdff675143b3b9b15d9f93d2e618.93c2cc329b1fe1893cfbc28c56960522->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf1/744bcdff675143b3b9b15d9f93d2e618-top, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/054f69c06f126aad97b290410d5d6188/cf1/testSplitTableRegionUnevenDaughter=93c2cc329b1fe1893cfbc28c56960522-e4b432d539b1457f9648cdf383c363c9, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/054f69c06f126aad97b290410d5d6188/cf1/testSplitTableRegionUnevenDaughter=93c2cc329b1fe1893cfbc28c56960522-a9634e7ee1844806a05a96cd8e89dd4d, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/054f69c06f126aad97b290410d5d6188/cf1/testSplitTableRegionUnevenDaughter=93c2cc329b1fe1893cfbc28c56960522-427cc49d3a8e4682b202f4b0673d8806] to archive 2023-05-10 04:53:46,943 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1683694424955.054f69c06f126aad97b290410d5d6188.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-05-10 04:53:46,945 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=23, resume processing ppid=20 2023-05-10 04:53:46,945 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=23, ppid=20, state=SUCCESS; CloseRegionProcedure 19040b6538a6bdeed7c833cfb44614b6, server=jenkins-hbase4.apache.org,37823,1683694419534 in 251 msec 2023-05-10 04:53:46,945 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-05-10 04:53:46,947 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-05-10 04:53:46,948 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=20, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=19040b6538a6bdeed7c833cfb44614b6, UNASSIGN in 266 msec 2023-05-10 04:53:46,949 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1683694424955.054f69c06f126aad97b290410d5d6188.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/054f69c06f126aad97b290410d5d6188/cf1/9333fac35a6e43f2ae1eb05c2a8c43e2.93c2cc329b1fe1893cfbc28c56960522 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionUnevenDaughter/054f69c06f126aad97b290410d5d6188/cf1/9333fac35a6e43f2ae1eb05c2a8c43e2.93c2cc329b1fe1893cfbc28c56960522 2023-05-10 04:53:46,951 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1683694424955.054f69c06f126aad97b290410d5d6188.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/054f69c06f126aad97b290410d5d6188/cf1/6fc2425571254b1b8437fabc5c780c71.93c2cc329b1fe1893cfbc28c56960522 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionUnevenDaughter/054f69c06f126aad97b290410d5d6188/cf1/6fc2425571254b1b8437fabc5c780c71.93c2cc329b1fe1893cfbc28c56960522 2023-05-10 04:53:46,952 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1683694424955.054f69c06f126aad97b290410d5d6188.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/054f69c06f126aad97b290410d5d6188/cf1/be24ca36992745b7a1a3b25d9bc68d53.93c2cc329b1fe1893cfbc28c56960522 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionUnevenDaughter/054f69c06f126aad97b290410d5d6188/cf1/be24ca36992745b7a1a3b25d9bc68d53.93c2cc329b1fe1893cfbc28c56960522 2023-05-10 04:53:46,954 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1683694424955.054f69c06f126aad97b290410d5d6188.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/054f69c06f126aad97b290410d5d6188/cf1/744bcdff675143b3b9b15d9f93d2e618.93c2cc329b1fe1893cfbc28c56960522 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionUnevenDaughter/054f69c06f126aad97b290410d5d6188/cf1/744bcdff675143b3b9b15d9f93d2e618.93c2cc329b1fe1893cfbc28c56960522 2023-05-10 04:53:46,956 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1683694424955.054f69c06f126aad97b290410d5d6188.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/054f69c06f126aad97b290410d5d6188/cf1/testSplitTableRegionUnevenDaughter=93c2cc329b1fe1893cfbc28c56960522-e4b432d539b1457f9648cdf383c363c9 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionUnevenDaughter/054f69c06f126aad97b290410d5d6188/cf1/testSplitTableRegionUnevenDaughter=93c2cc329b1fe1893cfbc28c56960522-e4b432d539b1457f9648cdf383c363c9 2023-05-10 04:53:46,958 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1683694424955.054f69c06f126aad97b290410d5d6188.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/054f69c06f126aad97b290410d5d6188/cf1/testSplitTableRegionUnevenDaughter=93c2cc329b1fe1893cfbc28c56960522-a9634e7ee1844806a05a96cd8e89dd4d to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionUnevenDaughter/054f69c06f126aad97b290410d5d6188/cf1/testSplitTableRegionUnevenDaughter=93c2cc329b1fe1893cfbc28c56960522-a9634e7ee1844806a05a96cd8e89dd4d 2023-05-10 04:53:46,959 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1683694424955.054f69c06f126aad97b290410d5d6188.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/054f69c06f126aad97b290410d5d6188/cf1/testSplitTableRegionUnevenDaughter=93c2cc329b1fe1893cfbc28c56960522-427cc49d3a8e4682b202f4b0673d8806 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionUnevenDaughter/054f69c06f126aad97b290410d5d6188/cf1/testSplitTableRegionUnevenDaughter=93c2cc329b1fe1893cfbc28c56960522-427cc49d3a8e4682b202f4b0673d8806 2023-05-10 04:53:46,961 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1683694424955.054f69c06f126aad97b290410d5d6188.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/054f69c06f126aad97b290410d5d6188/cf2/b1cbcf3117244e049435fa8e701a7a80.93c2cc329b1fe1893cfbc28c56960522->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf2/b1cbcf3117244e049435fa8e701a7a80-top, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/054f69c06f126aad97b290410d5d6188/cf2/bd68bbeaf0094fd0a415560924da2bce.93c2cc329b1fe1893cfbc28c56960522->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf2/bd68bbeaf0094fd0a415560924da2bce-top, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/054f69c06f126aad97b290410d5d6188/cf2/76f6f9416941403596b6ff3caccb861c.93c2cc329b1fe1893cfbc28c56960522->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf2/76f6f9416941403596b6ff3caccb861c-top, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/054f69c06f126aad97b290410d5d6188/cf2/4ee8175dfe724214ad50b9f4f420f989.93c2cc329b1fe1893cfbc28c56960522->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf2/4ee8175dfe724214ad50b9f4f420f989-top, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/054f69c06f126aad97b290410d5d6188/cf2/testSplitTableRegionUnevenDaughter=93c2cc329b1fe1893cfbc28c56960522-a4dcfc1811054c03bee550c410a35fbb, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/054f69c06f126aad97b290410d5d6188/cf2/testSplitTableRegionUnevenDaughter=93c2cc329b1fe1893cfbc28c56960522-7a78adce76124917861cfd93394e25fd, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/054f69c06f126aad97b290410d5d6188/cf2/testSplitTableRegionUnevenDaughter=93c2cc329b1fe1893cfbc28c56960522-48782fe6bd7d4d8996815ff75cdfac23] to archive 2023-05-10 04:53:46,962 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1683694424955.054f69c06f126aad97b290410d5d6188.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-05-10 04:53:46,964 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1683694424955.054f69c06f126aad97b290410d5d6188.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/054f69c06f126aad97b290410d5d6188/cf2/b1cbcf3117244e049435fa8e701a7a80.93c2cc329b1fe1893cfbc28c56960522 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionUnevenDaughter/054f69c06f126aad97b290410d5d6188/cf2/b1cbcf3117244e049435fa8e701a7a80.93c2cc329b1fe1893cfbc28c56960522 2023-05-10 04:53:46,966 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1683694424955.054f69c06f126aad97b290410d5d6188.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/054f69c06f126aad97b290410d5d6188/cf2/bd68bbeaf0094fd0a415560924da2bce.93c2cc329b1fe1893cfbc28c56960522 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionUnevenDaughter/054f69c06f126aad97b290410d5d6188/cf2/bd68bbeaf0094fd0a415560924da2bce.93c2cc329b1fe1893cfbc28c56960522 2023-05-10 04:53:46,967 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1683694424955.054f69c06f126aad97b290410d5d6188.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/054f69c06f126aad97b290410d5d6188/cf2/76f6f9416941403596b6ff3caccb861c.93c2cc329b1fe1893cfbc28c56960522 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionUnevenDaughter/054f69c06f126aad97b290410d5d6188/cf2/76f6f9416941403596b6ff3caccb861c.93c2cc329b1fe1893cfbc28c56960522 2023-05-10 04:53:46,969 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1683694424955.054f69c06f126aad97b290410d5d6188.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/054f69c06f126aad97b290410d5d6188/cf2/4ee8175dfe724214ad50b9f4f420f989.93c2cc329b1fe1893cfbc28c56960522 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionUnevenDaughter/054f69c06f126aad97b290410d5d6188/cf2/4ee8175dfe724214ad50b9f4f420f989.93c2cc329b1fe1893cfbc28c56960522 2023-05-10 04:53:46,970 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1683694424955.054f69c06f126aad97b290410d5d6188.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/054f69c06f126aad97b290410d5d6188/cf2/testSplitTableRegionUnevenDaughter=93c2cc329b1fe1893cfbc28c56960522-a4dcfc1811054c03bee550c410a35fbb to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionUnevenDaughter/054f69c06f126aad97b290410d5d6188/cf2/testSplitTableRegionUnevenDaughter=93c2cc329b1fe1893cfbc28c56960522-a4dcfc1811054c03bee550c410a35fbb 2023-05-10 04:53:46,972 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1683694424955.054f69c06f126aad97b290410d5d6188.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/054f69c06f126aad97b290410d5d6188/cf2/testSplitTableRegionUnevenDaughter=93c2cc329b1fe1893cfbc28c56960522-7a78adce76124917861cfd93394e25fd to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionUnevenDaughter/054f69c06f126aad97b290410d5d6188/cf2/testSplitTableRegionUnevenDaughter=93c2cc329b1fe1893cfbc28c56960522-7a78adce76124917861cfd93394e25fd 2023-05-10 04:53:46,973 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1683694424955.054f69c06f126aad97b290410d5d6188.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/054f69c06f126aad97b290410d5d6188/cf2/testSplitTableRegionUnevenDaughter=93c2cc329b1fe1893cfbc28c56960522-48782fe6bd7d4d8996815ff75cdfac23 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionUnevenDaughter/054f69c06f126aad97b290410d5d6188/cf2/testSplitTableRegionUnevenDaughter=93c2cc329b1fe1893cfbc28c56960522-48782fe6bd7d4d8996815ff75cdfac23 2023-05-10 04:53:46,978 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionUnevenDaughter/054f69c06f126aad97b290410d5d6188/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-05-10 04:53:46,979 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-10 04:53:46,980 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionUnevenDaughter,26,1683694424955.054f69c06f126aad97b290410d5d6188. 2023-05-10 04:53:46,980 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 054f69c06f126aad97b290410d5d6188: 2023-05-10 04:53:46,983 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 054f69c06f126aad97b290410d5d6188 2023-05-10 04:53:46,984 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=21 updating hbase:meta row=054f69c06f126aad97b290410d5d6188, regionState=CLOSED 2023-05-10 04:53:46,984 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,26,1683694424955.054f69c06f126aad97b290410d5d6188.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1683694426984"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683694426984"}]},"ts":"1683694426984"} 2023-05-10 04:53:46,985 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-05-10 04:53:46,988 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=22, resume processing ppid=21 2023-05-10 04:53:46,989 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=22, ppid=21, state=SUCCESS; CloseRegionProcedure 054f69c06f126aad97b290410d5d6188, server=jenkins-hbase4.apache.org,37823,1683694419534 in 303 msec 2023-05-10 04:53:46,991 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=21, resume processing ppid=19 2023-05-10 04:53:46,991 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=21, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=054f69c06f126aad97b290410d5d6188, UNASSIGN in 310 msec 2023-05-10 04:53:46,992 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683694426992"}]},"ts":"1683694426992"} 2023-05-10 04:53:46,994 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=DISABLED in hbase:meta 2023-05-10 04:53:46,996 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionUnevenDaughter to state=DISABLED 2023-05-10 04:53:46,998 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter in 335 msec 2023-05-10 04:53:47,286 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(1227): Checking to see if procedure is done pid=19 2023-05-10 04:53:47,287 INFO [Listener at localhost/45007] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionUnevenDaughter, procId: 19 completed 2023-05-10 04:53:47,291 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete testSplitTableRegionUnevenDaughter 2023-05-10 04:53:47,298 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] procedure2.ProcedureExecutor(1029): Stored pid=24, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:53:47,300 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=24, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:53:47,302 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=24, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:53:47,303 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(1227): Checking to see if procedure is done pid=24 2023-05-10 04:53:47,309 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522 2023-05-10 04:53:47,309 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionUnevenDaughter/19040b6538a6bdeed7c833cfb44614b6 2023-05-10 04:53:47,309 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionUnevenDaughter/054f69c06f126aad97b290410d5d6188 2023-05-10 04:53:47,315 DEBUG [HFileArchiver-5] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionUnevenDaughter/054f69c06f126aad97b290410d5d6188/cf1, FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionUnevenDaughter/054f69c06f126aad97b290410d5d6188/cf2, FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionUnevenDaughter/054f69c06f126aad97b290410d5d6188/recovered.edits] 2023-05-10 04:53:47,315 DEBUG [HFileArchiver-3] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf1, FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf2, FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/recovered.edits] 2023-05-10 04:53:47,315 DEBUG [HFileArchiver-4] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionUnevenDaughter/19040b6538a6bdeed7c833cfb44614b6/cf1, FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionUnevenDaughter/19040b6538a6bdeed7c833cfb44614b6/cf2, FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionUnevenDaughter/19040b6538a6bdeed7c833cfb44614b6/recovered.edits] 2023-05-10 04:53:47,319 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionUnevenDaughter/19040b6538a6bdeed7c833cfb44614b6/cf1/b9b226020897466e9944b392670962f2 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionUnevenDaughter/19040b6538a6bdeed7c833cfb44614b6/cf1/b9b226020897466e9944b392670962f2 2023-05-10 04:53:47,320 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf1/427cc49d3a8e4682b202f4b0673d8806 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf1/427cc49d3a8e4682b202f4b0673d8806 2023-05-10 04:53:47,320 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionUnevenDaughter/054f69c06f126aad97b290410d5d6188/cf1/564780056d5d4ccf975765c3601c012c to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionUnevenDaughter/054f69c06f126aad97b290410d5d6188/cf1/564780056d5d4ccf975765c3601c012c 2023-05-10 04:53:47,322 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf1/6fc2425571254b1b8437fabc5c780c71 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf1/6fc2425571254b1b8437fabc5c780c71 2023-05-10 04:53:47,323 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionUnevenDaughter/19040b6538a6bdeed7c833cfb44614b6/cf2/d185e4b0cdc3444ea290b89728d398a0 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionUnevenDaughter/19040b6538a6bdeed7c833cfb44614b6/cf2/d185e4b0cdc3444ea290b89728d398a0 2023-05-10 04:53:47,324 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf1/744bcdff675143b3b9b15d9f93d2e618 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf1/744bcdff675143b3b9b15d9f93d2e618 2023-05-10 04:53:47,324 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionUnevenDaughter/054f69c06f126aad97b290410d5d6188/cf2/a3e2994715ee4e608e3cf09a744ae646 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionUnevenDaughter/054f69c06f126aad97b290410d5d6188/cf2/a3e2994715ee4e608e3cf09a744ae646 2023-05-10 04:53:47,326 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf1/9333fac35a6e43f2ae1eb05c2a8c43e2 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf1/9333fac35a6e43f2ae1eb05c2a8c43e2 2023-05-10 04:53:47,331 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionUnevenDaughter/19040b6538a6bdeed7c833cfb44614b6/recovered.edits/90.seqid to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionUnevenDaughter/19040b6538a6bdeed7c833cfb44614b6/recovered.edits/90.seqid 2023-05-10 04:53:47,332 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf1/a9634e7ee1844806a05a96cd8e89dd4d to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf1/a9634e7ee1844806a05a96cd8e89dd4d 2023-05-10 04:53:47,332 DEBUG [HFileArchiver-4] backup.HFileArchiver(596): Deleted hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionUnevenDaughter/19040b6538a6bdeed7c833cfb44614b6 2023-05-10 04:53:47,333 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionUnevenDaughter/054f69c06f126aad97b290410d5d6188/recovered.edits/90.seqid to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionUnevenDaughter/054f69c06f126aad97b290410d5d6188/recovered.edits/90.seqid 2023-05-10 04:53:47,333 DEBUG [HFileArchiver-5] backup.HFileArchiver(596): Deleted hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionUnevenDaughter/054f69c06f126aad97b290410d5d6188 2023-05-10 04:53:47,333 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf1/be24ca36992745b7a1a3b25d9bc68d53 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf1/be24ca36992745b7a1a3b25d9bc68d53 2023-05-10 04:53:47,335 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf1/e4b432d539b1457f9648cdf383c363c9 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf1/e4b432d539b1457f9648cdf383c363c9 2023-05-10 04:53:47,338 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf2/48782fe6bd7d4d8996815ff75cdfac23 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf2/48782fe6bd7d4d8996815ff75cdfac23 2023-05-10 04:53:47,339 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf2/4ee8175dfe724214ad50b9f4f420f989 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf2/4ee8175dfe724214ad50b9f4f420f989 2023-05-10 04:53:47,341 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf2/76f6f9416941403596b6ff3caccb861c to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf2/76f6f9416941403596b6ff3caccb861c 2023-05-10 04:53:47,342 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf2/7a78adce76124917861cfd93394e25fd to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf2/7a78adce76124917861cfd93394e25fd 2023-05-10 04:53:47,344 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf2/a4dcfc1811054c03bee550c410a35fbb to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf2/a4dcfc1811054c03bee550c410a35fbb 2023-05-10 04:53:47,345 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf2/b1cbcf3117244e049435fa8e701a7a80 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf2/b1cbcf3117244e049435fa8e701a7a80 2023-05-10 04:53:47,347 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf2/bd68bbeaf0094fd0a415560924da2bce to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/cf2/bd68bbeaf0094fd0a415560924da2bce 2023-05-10 04:53:47,350 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/recovered.edits/85.seqid to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522/recovered.edits/85.seqid 2023-05-10 04:53:47,351 DEBUG [HFileArchiver-3] backup.HFileArchiver(596): Deleted hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionUnevenDaughter/93c2cc329b1fe1893cfbc28c56960522 2023-05-10 04:53:47,351 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionUnevenDaughter regions 2023-05-10 04:53:47,354 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=24, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:53:47,379 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionUnevenDaughter from hbase:meta 2023-05-10 04:53:47,405 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(1227): Checking to see if procedure is done pid=24 2023-05-10 04:53:47,424 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionUnevenDaughter' descriptor. 2023-05-10 04:53:47,426 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=24, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:53:47,426 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionUnevenDaughter' from region states. 2023-05-10 04:53:47,426 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter,,1683694422380.93c2cc329b1fe1893cfbc28c56960522.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1683694427426"}]},"ts":"9223372036854775807"} 2023-05-10 04:53:47,426 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter,,1683694424955.19040b6538a6bdeed7c833cfb44614b6.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1683694427426"}]},"ts":"9223372036854775807"} 2023-05-10 04:53:47,426 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter,26,1683694424955.054f69c06f126aad97b290410d5d6188.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1683694427426"}]},"ts":"9223372036854775807"} 2023-05-10 04:53:47,430 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-05-10 04:53:47,430 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 93c2cc329b1fe1893cfbc28c56960522, NAME => 'testSplitTableRegionUnevenDaughter,,1683694422380.93c2cc329b1fe1893cfbc28c56960522.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 19040b6538a6bdeed7c833cfb44614b6, NAME => 'testSplitTableRegionUnevenDaughter,,1683694424955.19040b6538a6bdeed7c833cfb44614b6.', STARTKEY => '', ENDKEY => '26'}, {ENCODED => 054f69c06f126aad97b290410d5d6188, NAME => 'testSplitTableRegionUnevenDaughter,26,1683694424955.054f69c06f126aad97b290410d5d6188.', STARTKEY => '26', ENDKEY => ''}] 2023-05-10 04:53:47,430 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionUnevenDaughter' as deleted. 2023-05-10 04:53:47,430 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1683694427430"}]},"ts":"9223372036854775807"} 2023-05-10 04:53:47,432 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionUnevenDaughter state from META 2023-05-10 04:53:47,435 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=24, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:53:47,436 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter in 142 msec 2023-05-10 04:53:47,607 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(1227): Checking to see if procedure is done pid=24 2023-05-10 04:53:47,607 INFO [Listener at localhost/45007] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionUnevenDaughter, procId: 24 completed 2023-05-10 04:53:47,624 INFO [Listener at localhost/45007] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionUnevenDaughter Thread=393 (was 360) Potentially hanging thread: HFileArchiver-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:2;jenkins-hbase4:37823-shortCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4a1c5cf4-shared-pool-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-515825336_15 at /127.0.0.1:55428 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-515825336_15 at /127.0.0.1:47324 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x248fdd40-shared-pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x248fdd40-shared-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-515825336_15 at /127.0.0.1:47634 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-515825336_15 at /127.0.0.1:47622 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-515825336_15 at /127.0.0.1:47266 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-2 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x248fdd40-metaLookup-shared--pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x248fdd40-metaLookup-shared--pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-515825336_15 at /127.0.0.1:52870 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-515825336_15 at /127.0.0.1:55438 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-3 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-515825336_15 at /127.0.0.1:55656 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-515825336_15 at /127.0.0.1:47638 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x248fdd40-shared-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4a1c5cf4-shared-pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-515825336_15 at /127.0.0.1:55458 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-515825336_15 at /127.0.0.1:55696 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: (jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RSProcedureDispatcher-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4a1c5cf4-shared-pool-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x248fdd40-shared-pool-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-515825336_15 at /127.0.0.1:55674 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-515825336_15 at /127.0.0.1:47286 [Waiting for operation #13] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Timer for 'HBase' metrics system java.lang.Object.wait(Native Method) java.util.TimerThread.mainLoop(Timer.java:552) java.util.TimerThread.run(Timer.java:505) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-515825336_15 at /127.0.0.1:47388 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-515825336_15 at /127.0.0.1:55672 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4a1c5cf4-shared-pool-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-515825336_15 at /127.0.0.1:52854 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=831 (was 695) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=348 (was 344) - SystemLoadAverage LEAK? -, ProcessCount=175 (was 175), AvailableMemoryMB=9729 (was 9782) 2023-05-10 04:53:47,639 INFO [Listener at localhost/45007] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitWithoutPONR Thread=393, OpenFileDescriptor=831, MaxFileDescriptor=60000, SystemLoadAverage=348, ProcessCount=175, AvailableMemoryMB=9728 2023-05-10 04:53:47,639 WARN [Listener at localhost/45007] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-05-10 04:53:47,649 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-05-10 04:53:47,649 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-05-10 04:53:47,651 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(1029): Stored pid=25, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitWithoutPONR 2023-05-10 04:53:47,653 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_PRE_OPERATION 2023-05-10 04:53:47,654 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-05-10 04:53:47,657 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:47,657 DEBUG [HFileArchiver-6] backup.HFileArchiver(153): Directory hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66 empty. 2023-05-10 04:53:47,658 DEBUG [HFileArchiver-6] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:47,658 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitWithoutPONR regions 2023-05-10 04:53:47,672 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitWithoutPONR/.tabledesc/.tableinfo.0000000001 2023-05-10 04:53:47,674 INFO [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(7675): creating {ENCODED => 6c70954228a0bbe97dfdc76861367c66, NAME => 'testSplitWithoutPONR,,1683694427650.6c70954228a0bbe97dfdc76861367c66.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitWithoutPONR', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp 2023-05-10 04:53:47,688 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(866): Instantiated testSplitWithoutPONR,,1683694427650.6c70954228a0bbe97dfdc76861367c66.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-10 04:53:47,688 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1604): Closing 6c70954228a0bbe97dfdc76861367c66, disabling compactions & flushes 2023-05-10 04:53:47,688 INFO [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1626): Closing region testSplitWithoutPONR,,1683694427650.6c70954228a0bbe97dfdc76861367c66. 2023-05-10 04:53:47,688 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitWithoutPONR,,1683694427650.6c70954228a0bbe97dfdc76861367c66. 2023-05-10 04:53:47,688 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1714): Acquired close lock on testSplitWithoutPONR,,1683694427650.6c70954228a0bbe97dfdc76861367c66. after waiting 0 ms 2023-05-10 04:53:47,688 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1724): Updates disabled for region testSplitWithoutPONR,,1683694427650.6c70954228a0bbe97dfdc76861367c66. 2023-05-10 04:53:47,688 INFO [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1838): Closed testSplitWithoutPONR,,1683694427650.6c70954228a0bbe97dfdc76861367c66. 2023-05-10 04:53:47,688 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1558): Region close journal for 6c70954228a0bbe97dfdc76861367c66: 2023-05-10 04:53:47,691 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_ADD_TO_META 2023-05-10 04:53:47,692 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitWithoutPONR,,1683694427650.6c70954228a0bbe97dfdc76861367c66.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1683694427692"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683694427692"}]},"ts":"1683694427692"} 2023-05-10 04:53:47,694 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-05-10 04:53:47,695 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-05-10 04:53:47,695 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683694427695"}]},"ts":"1683694427695"} 2023-05-10 04:53:47,696 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=ENABLING in hbase:meta 2023-05-10 04:53:47,699 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-05-10 04:53:47,700 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-05-10 04:53:47,700 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-05-10 04:53:47,700 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-05-10 04:53:47,700 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-05-10 04:53:47,701 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=26, ppid=25, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=6c70954228a0bbe97dfdc76861367c66, ASSIGN}] 2023-05-10 04:53:47,703 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=26, ppid=25, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=6c70954228a0bbe97dfdc76861367c66, ASSIGN 2023-05-10 04:53:47,703 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=26, ppid=25, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=6c70954228a0bbe97dfdc76861367c66, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,37823,1683694419534; forceNewPlan=false, retain=false 2023-05-10 04:53:47,854 INFO [jenkins-hbase4:37903] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-05-10 04:53:47,855 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=26 updating hbase:meta row=6c70954228a0bbe97dfdc76861367c66, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:47,855 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1683694427650.6c70954228a0bbe97dfdc76861367c66.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1683694427855"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683694427855"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683694427855"}]},"ts":"1683694427855"} 2023-05-10 04:53:47,857 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=27, ppid=26, state=RUNNABLE; OpenRegionProcedure 6c70954228a0bbe97dfdc76861367c66, server=jenkins-hbase4.apache.org,37823,1683694419534}] 2023-05-10 04:53:48,013 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitWithoutPONR,,1683694427650.6c70954228a0bbe97dfdc76861367c66. 2023-05-10 04:53:48,013 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 6c70954228a0bbe97dfdc76861367c66, NAME => 'testSplitWithoutPONR,,1683694427650.6c70954228a0bbe97dfdc76861367c66.', STARTKEY => '', ENDKEY => ''} 2023-05-10 04:53:48,014 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-05-10 04:53:48,014 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitWithoutPONR 6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:48,014 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitWithoutPONR,,1683694427650.6c70954228a0bbe97dfdc76861367c66.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-10 04:53:48,014 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:48,014 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:48,015 INFO [StoreOpener-6c70954228a0bbe97dfdc76861367c66-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:48,017 DEBUG [StoreOpener-6c70954228a0bbe97dfdc76861367c66-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf1 2023-05-10 04:53:48,017 DEBUG [StoreOpener-6c70954228a0bbe97dfdc76861367c66-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf1 2023-05-10 04:53:48,017 INFO [StoreOpener-6c70954228a0bbe97dfdc76861367c66-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6c70954228a0bbe97dfdc76861367c66 columnFamilyName cf1 2023-05-10 04:53:48,018 INFO [StoreOpener-6c70954228a0bbe97dfdc76861367c66-1] regionserver.HStore(310): Store=6c70954228a0bbe97dfdc76861367c66/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-10 04:53:48,018 INFO [StoreOpener-6c70954228a0bbe97dfdc76861367c66-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:48,020 DEBUG [StoreOpener-6c70954228a0bbe97dfdc76861367c66-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf2 2023-05-10 04:53:48,020 DEBUG [StoreOpener-6c70954228a0bbe97dfdc76861367c66-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf2 2023-05-10 04:53:48,021 INFO [StoreOpener-6c70954228a0bbe97dfdc76861367c66-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6c70954228a0bbe97dfdc76861367c66 columnFamilyName cf2 2023-05-10 04:53:48,022 INFO [StoreOpener-6c70954228a0bbe97dfdc76861367c66-1] regionserver.HStore(310): Store=6c70954228a0bbe97dfdc76861367c66/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-10 04:53:48,023 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:48,024 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:48,026 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitWithoutPONR descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-05-10 04:53:48,028 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:48,030 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-05-10 04:53:48,031 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 6c70954228a0bbe97dfdc76861367c66; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70718223, jitterRate=0.05378364026546478}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-05-10 04:53:48,031 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 6c70954228a0bbe97dfdc76861367c66: 2023-05-10 04:53:48,032 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2334): Post open deploy tasks for testSplitWithoutPONR,,1683694427650.6c70954228a0bbe97dfdc76861367c66., pid=27, masterSystemTime=1683694428009 2023-05-10 04:53:48,040 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2361): Finished post open deploy task for testSplitWithoutPONR,,1683694427650.6c70954228a0bbe97dfdc76861367c66. 2023-05-10 04:53:48,040 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitWithoutPONR,,1683694427650.6c70954228a0bbe97dfdc76861367c66. 2023-05-10 04:53:48,041 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=26 updating hbase:meta row=6c70954228a0bbe97dfdc76861367c66, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:48,041 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitWithoutPONR,,1683694427650.6c70954228a0bbe97dfdc76861367c66.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1683694428041"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1683694428041"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1683694428041"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683694428041"}]},"ts":"1683694428041"} 2023-05-10 04:53:48,045 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=27, resume processing ppid=26 2023-05-10 04:53:48,045 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=27, ppid=26, state=SUCCESS; OpenRegionProcedure 6c70954228a0bbe97dfdc76861367c66, server=jenkins-hbase4.apache.org,37823,1683694419534 in 186 msec 2023-05-10 04:53:48,048 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=26, resume processing ppid=25 2023-05-10 04:53:48,048 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=26, ppid=25, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=6c70954228a0bbe97dfdc76861367c66, ASSIGN in 344 msec 2023-05-10 04:53:48,049 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-05-10 04:53:48,049 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683694428049"}]},"ts":"1683694428049"} 2023-05-10 04:53:48,051 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=ENABLED in hbase:meta 2023-05-10 04:53:48,053 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_POST_OPERATION 2023-05-10 04:53:48,055 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR in 404 msec 2023-05-10 04:53:48,164 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-10 04:53:48,166 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-05-10 04:53:48,166 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-05-10 04:53:48,166 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:53:48,167 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-05-10 04:53:48,167 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-05-10 04:53:48,168 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-10 04:53:48,168 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-10 04:53:48,171 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:53:48,171 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:53:48,171 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:53:48,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:53:48,171 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:53:48,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:53:48,171 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:48,171 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:53:48,171 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:53:48,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:53:48,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-10 04:53:48,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-10 04:53:48,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-10 04:53:48,172 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:48,172 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-10 04:53:48,172 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-10 04:53:48,172 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-10 04:53:48,172 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-10 04:53:48,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:53:48,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:53:48,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-10 04:53:48,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:53:48,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-10 04:53:48,173 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-05-10 04:53:48,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-10 04:53:48,173 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-05-10 04:53:48,173 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-05-10 04:53:48,173 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-05-10 04:53:48,173 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-05-10 04:53:48,173 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:53:48,173 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-05-10 04:53:48,173 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-05-10 04:53:48,174 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:53:48,176 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-05-10 04:53:48,176 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-05-10 04:53:48,176 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-05-10 04:53:48,178 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-05-10 04:53:48,178 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-05-10 04:53:48,178 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:53:48,178 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1683694427650.6c70954228a0bbe97dfdc76861367c66. 2023-05-10 04:53:48,177 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-10 04:53:48,178 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-10 04:53:48,178 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1683694427650.6c70954228a0bbe97dfdc76861367c66. started... 2023-05-10 04:53:48,178 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-05-10 04:53:48,178 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-05-10 04:53:48,178 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37995,1683694419499' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-05-10 04:53:48,178 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36665,1683694419452' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-05-10 04:53:48,179 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 6c70954228a0bbe97dfdc76861367c66 2/2 column families, dataSize=120 B heapSize=928 B 2023-05-10 04:53:48,181 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-10 04:53:48,181 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-10 04:53:48,181 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-10 04:53:48,181 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:53:48,181 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-10 04:53:48,182 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:53:48,193 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/.tmp/cf1/692164c792dc453096c76676d73fe8e8 2023-05-10 04:53:48,209 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/.tmp/cf2/5b75cfdc8cfb4ced978b992b5b43f1be 2023-05-10 04:53:48,215 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/.tmp/cf1/692164c792dc453096c76676d73fe8e8 as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf1/692164c792dc453096c76676d73fe8e8 2023-05-10 04:53:48,220 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf1/692164c792dc453096c76676d73fe8e8, entries=2, sequenceid=6, filesize=4.8 K 2023-05-10 04:53:48,221 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/.tmp/cf2/5b75cfdc8cfb4ced978b992b5b43f1be as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf2/5b75cfdc8cfb4ced978b992b5b43f1be 2023-05-10 04:53:48,226 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf2/5b75cfdc8cfb4ced978b992b5b43f1be, entries=2, sequenceid=6, filesize=4.8 K 2023-05-10 04:53:48,227 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 6c70954228a0bbe97dfdc76861367c66 in 48ms, sequenceid=6, compaction requested=false 2023-05-10 04:53:48,228 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitWithoutPONR' 2023-05-10 04:53:48,228 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 6c70954228a0bbe97dfdc76861367c66: 2023-05-10 04:53:48,228 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1683694427650.6c70954228a0bbe97dfdc76861367c66. 2023-05-10 04:53:48,228 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-10 04:53:48,228 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-10 04:53:48,229 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:48,229 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-05-10 04:53:48,229 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37823,1683694419534' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-05-10 04:53:48,230 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:48,230 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-10 04:53:48,230 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:48,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:53:48,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:53:48,231 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-10 04:53:48,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:53:48,231 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:53:48,231 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:53:48,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-10 04:53:48,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:48,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:48,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:48,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:53:48,233 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37823,1683694419534' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-05-10 04:53:48,233 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7e6171db[Count = 0] remaining members to acquire global barrier 2023-05-10 04:53:48,234 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-05-10 04:53:48,234 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-10 04:53:48,235 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-10 04:53:48,235 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-10 04:53:48,235 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-10 04:53:48,235 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-10 04:53:48,235 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-10 04:53:48,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-10 04:53:48,235 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-10 04:53:48,235 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-05-10 04:53:48,235 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-05-10 04:53:48,235 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:48,235 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-10 04:53:48,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-10 04:53:48,235 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,37995,1683694419499' in zk 2023-05-10 04:53:48,236 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-05-10 04:53:48,236 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-05-10 04:53:48,236 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,37823,1683694419534' in zk 2023-05-10 04:53:48,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-10 04:53:48,236 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-05-10 04:53:48,236 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-05-10 04:53:48,236 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,36665,1683694419452' in zk 2023-05-10 04:53:48,239 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-05-10 04:53:48,239 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:53:48,239 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-05-10 04:53:48,240 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-05-10 04:53:48,240 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:53:48,240 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-05-10 04:53:48,241 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-05-10 04:53:48,241 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:48,241 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:53:48,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:48,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:53:48,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:53:48,241 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-05-10 04:53:48,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:53:48,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:53:48,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-10 04:53:48,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:48,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:48,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:48,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:53:48,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-10 04:53:48,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:48,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:48,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:48,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,37823,1683694419534': 2023-05-10 04:53:48,246 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37823,1683694419534' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-05-10 04:53:48,246 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-05-10 04:53:48,246 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-10 04:53:48,246 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-10 04:53:48,246 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-05-10 04:53:48,246 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-10 04:53:48,248 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-10 04:53:48,248 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-10 04:53:48,248 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-10 04:53:48,248 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-10 04:53:48,248 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-10 04:53:48,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-10 04:53:48,248 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-10 04:53:48,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-10 04:53:48,248 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-10 04:53:48,248 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:53:48,248 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:53:48,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-10 04:53:48,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-10 04:53:48,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:53:48,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:53:48,248 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:53:48,249 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:48,249 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:53:48,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:53:48,249 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:53:48,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:53:48,249 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:53:48,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:53:48,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-10 04:53:48,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-10 04:53:48,249 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:48,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:53:48,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-10 04:53:48,250 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:48,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-10 04:53:48,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:53:48,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-10 04:53:48,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:48,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:48,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:48,252 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:48,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:53:48,252 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:48,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-10 04:53:48,253 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:48,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:48,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:48,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:48,259 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:48,259 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:53:48,259 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:53:48,259 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:53:48,259 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:53:48,259 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:53:48,259 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:53:48,259 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-10 04:53:48,259 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:53:48,259 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-05-10 04:53:48,259 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-10 04:53:48,259 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:53:48,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:53:48,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:53:48,259 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:53:48,260 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:53:48,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:53:48,259 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:48,259 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:53:48,260 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:48,260 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-10 04:53:48,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:53:48,261 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:53:48,260 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-05-10 04:53:48,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:53:48,261 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:48,261 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:53:48,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:53:48,261 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-10 04:53:48,261 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-10 04:53:48,262 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:48,262 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:48,262 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-10 04:53:48,262 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-10 04:53:48,362 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-10 04:53:48,363 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-10 04:53:48,382 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-10 04:53:48,384 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-05-10 04:53:48,384 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-05-10 04:53:48,384 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-05-10 04:53:48,384 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:53:48,385 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-05-10 04:53:48,385 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-05-10 04:53:48,385 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-10 04:53:48,385 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-10 04:53:48,387 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:48,387 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:53:48,387 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:53:48,387 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:53:48,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:53:48,387 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:53:48,387 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:53:48,387 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:48,387 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-10 04:53:48,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:53:48,387 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:53:48,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:53:48,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-10 04:53:48,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-10 04:53:48,388 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-10 04:53:48,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-10 04:53:48,388 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-10 04:53:48,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:53:48,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-10 04:53:48,388 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-10 04:53:48,388 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-05-10 04:53:48,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:53:48,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-10 04:53:48,389 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-05-10 04:53:48,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:53:48,389 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-10 04:53:48,389 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-05-10 04:53:48,389 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-05-10 04:53:48,389 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-05-10 04:53:48,389 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-05-10 04:53:48,389 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-05-10 04:53:48,389 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-05-10 04:53:48,389 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-05-10 04:53:48,389 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:53:48,389 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-05-10 04:53:48,389 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-05-10 04:53:48,390 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:53:48,390 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-05-10 04:53:48,391 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:53:48,390 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-05-10 04:53:48,391 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-05-10 04:53:48,391 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-05-10 04:53:48,392 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-05-10 04:53:48,391 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-05-10 04:53:48,392 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36665,1683694419452' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-05-10 04:53:48,392 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37995,1683694419499' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-05-10 04:53:48,392 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-10 04:53:48,392 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-10 04:53:48,392 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1683694427650.6c70954228a0bbe97dfdc76861367c66. 2023-05-10 04:53:48,392 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1683694427650.6c70954228a0bbe97dfdc76861367c66. started... 2023-05-10 04:53:48,392 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 6c70954228a0bbe97dfdc76861367c66 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-10 04:53:48,393 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-10 04:53:48,393 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-10 04:53:48,393 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-10 04:53:48,393 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-10 04:53:48,394 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:53:48,394 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:53:48,403 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/.tmp/cf1/94f3fbf75ed04880bf66a2015ae1796f 2023-05-10 04:53:48,419 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/.tmp/cf2/51d8a67670c645628aeb50ddbfc1a8dc 2023-05-10 04:53:48,425 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/.tmp/cf1/94f3fbf75ed04880bf66a2015ae1796f as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf1/94f3fbf75ed04880bf66a2015ae1796f 2023-05-10 04:53:48,430 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf1/94f3fbf75ed04880bf66a2015ae1796f, entries=10, sequenceid=19, filesize=5.0 K 2023-05-10 04:53:48,431 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/.tmp/cf2/51d8a67670c645628aeb50ddbfc1a8dc as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf2/51d8a67670c645628aeb50ddbfc1a8dc 2023-05-10 04:53:48,437 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf2/51d8a67670c645628aeb50ddbfc1a8dc, entries=10, sequenceid=19, filesize=5.0 K 2023-05-10 04:53:48,438 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 6c70954228a0bbe97dfdc76861367c66 in 46ms, sequenceid=19, compaction requested=false 2023-05-10 04:53:48,438 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 6c70954228a0bbe97dfdc76861367c66: 2023-05-10 04:53:48,438 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1683694427650.6c70954228a0bbe97dfdc76861367c66. 2023-05-10 04:53:48,438 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-10 04:53:48,438 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-10 04:53:48,438 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:48,438 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-05-10 04:53:48,438 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37823,1683694419534' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-05-10 04:53:48,440 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-10 04:53:48,440 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:48,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:48,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:53:48,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:53:48,440 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-10 04:53:48,440 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:53:48,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:53:48,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:53:48,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-10 04:53:48,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:48,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:48,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:48,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:53:48,442 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37823,1683694419534' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-05-10 04:53:48,442 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2152fcdd[Count = 0] remaining members to acquire global barrier 2023-05-10 04:53:48,442 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-05-10 04:53:48,443 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-10 04:53:48,445 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-10 04:53:48,445 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-10 04:53:48,445 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-10 04:53:48,445 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-10 04:53:48,445 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-10 04:53:48,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-10 04:53:48,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-10 04:53:48,445 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:48,445 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-05-10 04:53:48,445 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-05-10 04:53:48,445 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-10 04:53:48,445 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,37995,1683694419499' in zk 2023-05-10 04:53:48,445 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-05-10 04:53:48,445 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-05-10 04:53:48,445 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-10 04:53:48,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-10 04:53:48,445 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,37823,1683694419534' in zk 2023-05-10 04:53:48,446 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-05-10 04:53:48,446 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-05-10 04:53:48,446 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,36665,1683694419452' in zk 2023-05-10 04:53:48,447 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-05-10 04:53:48,447 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:53:48,447 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-05-10 04:53:48,450 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-05-10 04:53:48,450 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:48,450 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:53:48,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:48,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:53:48,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:53:48,451 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-05-10 04:53:48,452 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-05-10 04:53:48,452 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:53:48,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:53:48,452 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-05-10 04:53:48,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:53:48,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-10 04:53:48,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:48,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:48,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:48,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:53:48,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-10 04:53:48,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:48,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:48,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:48,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,37823,1683694419534': 2023-05-10 04:53:48,457 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37823,1683694419534' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-05-10 04:53:48,457 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-05-10 04:53:48,457 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-10 04:53:48,457 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-10 04:53:48,457 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-05-10 04:53:48,457 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-10 04:53:48,459 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-10 04:53:48,459 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-10 04:53:48,459 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-10 04:53:48,459 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:53:48,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-10 04:53:48,459 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-10 04:53:48,459 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:53:48,459 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-10 04:53:48,459 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-10 04:53:48,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-10 04:53:48,460 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:53:48,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:53:48,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:53:48,460 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-10 04:53:48,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-10 04:53:48,460 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:48,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-10 04:53:48,460 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:53:48,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:53:48,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:53:48,460 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:53:48,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:53:48,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-10 04:53:48,461 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:53:48,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-10 04:53:48,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:53:48,461 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:48,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-10 04:53:48,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:53:48,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-10 04:53:48,461 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:48,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-10 04:53:48,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:48,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:48,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:48,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:53:48,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-10 04:53:48,463 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:48,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:48,464 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:48,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:48,464 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:48,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:48,468 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:53:48,468 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:53:48,468 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:53:48,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:53:48,468 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:48,468 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:53:48,468 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-05-10 04:53:48,468 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-10 04:53:48,468 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:53:48,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:53:48,468 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-10 04:53:48,468 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:53:48,469 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-05-10 04:53:48,469 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-10 04:53:48,469 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:53:48,468 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:53:48,468 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:53:48,469 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:53:48,469 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:53:48,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:53:48,468 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:48,469 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:48,469 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-10 04:53:48,469 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:48,470 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-10 04:53:48,470 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:48,470 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:48,470 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-10 04:53:48,470 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-10 04:53:48,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:53:48,469 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:53:48,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:53:48,470 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:53:48,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:53:48,485 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-05-10 04:53:48,485 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint Metrics about HBase RegionObservers 2023-05-10 04:53:48,486 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-10 04:53:48,486 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro Metrics about HBase RegionObservers 2023-05-10 04:53:48,569 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-10 04:53:48,571 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-10 04:53:48,590 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-10 04:53:48,591 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-05-10 04:53:48,591 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-05-10 04:53:48,591 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-05-10 04:53:48,592 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:53:48,592 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-05-10 04:53:48,592 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-05-10 04:53:48,593 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-10 04:53:48,593 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-10 04:53:48,594 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:48,594 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:53:48,594 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:53:48,594 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:53:48,594 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:53:48,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:53:48,594 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:53:48,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:53:48,595 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:53:48,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:53:48,595 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:48,595 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-10 04:53:48,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-10 04:53:48,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-10 04:53:48,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-10 04:53:48,595 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-10 04:53:48,595 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-10 04:53:48,595 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-10 04:53:48,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:53:48,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-10 04:53:48,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:53:48,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-10 04:53:48,596 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-05-10 04:53:48,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:53:48,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-10 04:53:48,596 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-05-10 04:53:48,596 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-05-10 04:53:48,596 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-05-10 04:53:48,596 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-05-10 04:53:48,596 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-05-10 04:53:48,596 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-05-10 04:53:48,596 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-05-10 04:53:48,596 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:53:48,596 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-05-10 04:53:48,597 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-05-10 04:53:48,596 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-05-10 04:53:48,597 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:53:48,597 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-05-10 04:53:48,598 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-05-10 04:53:48,598 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-05-10 04:53:48,598 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:53:48,597 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-10 04:53:48,599 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-05-10 04:53:48,599 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-10 04:53:48,599 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-05-10 04:53:48,599 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36665,1683694419452' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-05-10 04:53:48,598 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-05-10 04:53:48,598 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1683694427650.6c70954228a0bbe97dfdc76861367c66. 2023-05-10 04:53:48,599 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37995,1683694419499' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-05-10 04:53:48,599 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1683694427650.6c70954228a0bbe97dfdc76861367c66. started... 2023-05-10 04:53:48,599 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 6c70954228a0bbe97dfdc76861367c66 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-10 04:53:48,601 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-10 04:53:48,602 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-10 04:53:48,602 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-10 04:53:48,602 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:53:48,602 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-10 04:53:48,603 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:53:48,610 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/.tmp/cf1/1db12d8725064a0e8efea5e9b8d4ee97 2023-05-10 04:53:48,626 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/.tmp/cf2/8edaf1b5392e42598afb9d4e334cf824 2023-05-10 04:53:48,632 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/.tmp/cf1/1db12d8725064a0e8efea5e9b8d4ee97 as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf1/1db12d8725064a0e8efea5e9b8d4ee97 2023-05-10 04:53:48,638 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf1/1db12d8725064a0e8efea5e9b8d4ee97, entries=10, sequenceid=32, filesize=5.0 K 2023-05-10 04:53:48,639 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/.tmp/cf2/8edaf1b5392e42598afb9d4e334cf824 as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf2/8edaf1b5392e42598afb9d4e334cf824 2023-05-10 04:53:48,644 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf2/8edaf1b5392e42598afb9d4e334cf824, entries=10, sequenceid=32, filesize=5.0 K 2023-05-10 04:53:48,646 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 6c70954228a0bbe97dfdc76861367c66 in 46ms, sequenceid=32, compaction requested=true 2023-05-10 04:53:48,646 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 6c70954228a0bbe97dfdc76861367c66: 2023-05-10 04:53:48,646 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1683694427650.6c70954228a0bbe97dfdc76861367c66. 2023-05-10 04:53:48,646 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-10 04:53:48,646 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-10 04:53:48,646 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:48,646 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-05-10 04:53:48,646 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37823,1683694419534' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-05-10 04:53:48,648 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-10 04:53:48,648 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:48,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:48,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:53:48,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:53:48,648 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-10 04:53:48,648 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:53:48,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:53:48,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:53:48,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-10 04:53:48,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:48,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:48,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:48,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:53:48,650 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37823,1683694419534' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-05-10 04:53:48,651 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2306b677[Count = 0] remaining members to acquire global barrier 2023-05-10 04:53:48,651 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-05-10 04:53:48,651 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-10 04:53:48,652 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-10 04:53:48,652 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-10 04:53:48,652 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-10 04:53:48,652 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-10 04:53:48,652 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-10 04:53:48,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-10 04:53:48,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-10 04:53:48,652 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-05-10 04:53:48,652 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-05-10 04:53:48,652 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-05-10 04:53:48,652 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,37823,1683694419534' in zk 2023-05-10 04:53:48,653 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:48,653 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-10 04:53:48,652 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-10 04:53:48,652 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-05-10 04:53:48,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-10 04:53:48,653 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,37995,1683694419499' in zk 2023-05-10 04:53:48,653 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-05-10 04:53:48,653 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-05-10 04:53:48,653 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,36665,1683694419452' in zk 2023-05-10 04:53:48,655 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:48,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:48,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:53:48,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:53:48,655 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-05-10 04:53:48,655 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:53:48,655 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-05-10 04:53:48,656 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-05-10 04:53:48,656 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:53:48,656 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-05-10 04:53:48,657 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-05-10 04:53:48,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:53:48,657 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:53:48,658 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-05-10 04:53:48,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:53:48,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-10 04:53:48,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:48,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:48,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:48,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:53:48,659 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-10 04:53:48,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:48,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:48,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:48,660 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,37823,1683694419534': 2023-05-10 04:53:48,661 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37823,1683694419534' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-05-10 04:53:48,661 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-05-10 04:53:48,661 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-10 04:53:48,661 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-10 04:53:48,661 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-05-10 04:53:48,661 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-10 04:53:48,663 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-10 04:53:48,663 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-10 04:53:48,663 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-10 04:53:48,663 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-10 04:53:48,663 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:53:48,663 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-10 04:53:48,663 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:53:48,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-10 04:53:48,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:53:48,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:53:48,663 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-10 04:53:48,663 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:53:48,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-10 04:53:48,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-10 04:53:48,663 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:48,663 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-10 04:53:48,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-10 04:53:48,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:53:48,666 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:53:48,666 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:53:48,666 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:53:48,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:53:48,666 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:48,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:53:48,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:53:48,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-10 04:53:48,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-10 04:53:48,671 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:48,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-10 04:53:48,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-10 04:53:48,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:53:48,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-10 04:53:48,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:48,672 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:48,673 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:48,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:48,673 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:48,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:53:48,673 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:48,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-10 04:53:48,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:48,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:48,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:48,677 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:53:48,677 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:53:48,677 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:53:48,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:53:48,677 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:53:48,677 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:48,677 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-10 04:53:48,677 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:48,677 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:48,678 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-10 04:53:48,678 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:53:48,678 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:53:48,678 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:53:48,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:53:48,678 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:53:48,678 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-05-10 04:53:48,678 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-10 04:53:48,677 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:53:48,678 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:48,677 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:53:48,679 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-05-10 04:53:48,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:53:48,679 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-10 04:53:48,678 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-10 04:53:48,678 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:53:48,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:53:48,678 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:53:48,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:53:48,679 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:53:48,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:53:48,679 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:48,680 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:48,680 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-10 04:53:48,680 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-10 04:53:48,779 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-10 04:53:48,781 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-10 04:53:48,799 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-10 04:53:48,801 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-05-10 04:53:48,801 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-05-10 04:53:48,801 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-05-10 04:53:48,801 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:53:48,802 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-05-10 04:53:48,802 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-05-10 04:53:48,802 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-10 04:53:48,802 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-10 04:53:48,804 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:53:48,804 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:48,804 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:53:48,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:53:48,804 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:53:48,804 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:53:48,804 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:53:48,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:53:48,804 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:53:48,804 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:48,804 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-10 04:53:48,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:53:48,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-10 04:53:48,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-10 04:53:48,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-10 04:53:48,805 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-10 04:53:48,805 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-10 04:53:48,805 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-10 04:53:48,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:53:48,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-10 04:53:48,805 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-05-10 04:53:48,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:53:48,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:53:48,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-10 04:53:48,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-10 04:53:48,805 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-05-10 04:53:48,805 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-05-10 04:53:48,805 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-05-10 04:53:48,805 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-05-10 04:53:48,806 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-05-10 04:53:48,806 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:53:48,806 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-05-10 04:53:48,806 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-05-10 04:53:48,806 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-05-10 04:53:48,806 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-05-10 04:53:48,807 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-05-10 04:53:48,807 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37995,1683694419499' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-05-10 04:53:48,806 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-05-10 04:53:48,807 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:53:48,807 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-05-10 04:53:48,807 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-05-10 04:53:48,807 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-05-10 04:53:48,807 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:53:48,807 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-05-10 04:53:48,809 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36665,1683694419452' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-05-10 04:53:48,809 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-05-10 04:53:48,809 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-10 04:53:48,810 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-10 04:53:48,810 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1683694427650.6c70954228a0bbe97dfdc76861367c66. 2023-05-10 04:53:48,810 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1683694427650.6c70954228a0bbe97dfdc76861367c66. started... 2023-05-10 04:53:48,810 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 6c70954228a0bbe97dfdc76861367c66 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-10 04:53:48,813 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-10 04:53:48,816 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-10 04:53:48,816 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-10 04:53:48,816 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:53:48,816 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-10 04:53:48,816 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:53:48,822 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/.tmp/cf1/6caf2d6b1504485d93a71a5fd6c7407b 2023-05-10 04:53:48,837 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/.tmp/cf2/d5d0539ee2494349beff0d9e436cca78 2023-05-10 04:53:48,843 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/.tmp/cf1/6caf2d6b1504485d93a71a5fd6c7407b as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf1/6caf2d6b1504485d93a71a5fd6c7407b 2023-05-10 04:53:48,848 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf1/6caf2d6b1504485d93a71a5fd6c7407b, entries=10, sequenceid=45, filesize=5.0 K 2023-05-10 04:53:48,849 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/.tmp/cf2/d5d0539ee2494349beff0d9e436cca78 as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf2/d5d0539ee2494349beff0d9e436cca78 2023-05-10 04:53:48,855 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf2/d5d0539ee2494349beff0d9e436cca78, entries=10, sequenceid=45, filesize=5.0 K 2023-05-10 04:53:48,856 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 6c70954228a0bbe97dfdc76861367c66 in 46ms, sequenceid=45, compaction requested=true 2023-05-10 04:53:48,856 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 6c70954228a0bbe97dfdc76861367c66: 2023-05-10 04:53:48,856 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1683694427650.6c70954228a0bbe97dfdc76861367c66. 2023-05-10 04:53:48,856 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-10 04:53:48,856 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-10 04:53:48,856 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:48,856 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-05-10 04:53:48,856 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37823,1683694419534' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-05-10 04:53:48,858 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-10 04:53:48,858 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:48,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:48,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:53:48,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:53:48,858 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-10 04:53:48,858 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:53:48,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:53:48,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:53:48,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-10 04:53:48,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:48,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:48,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:48,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:53:48,860 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37823,1683694419534' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-05-10 04:53:48,860 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@31c032e4[Count = 0] remaining members to acquire global barrier 2023-05-10 04:53:48,860 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-05-10 04:53:48,860 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-10 04:53:48,862 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-10 04:53:48,862 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-10 04:53:48,862 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-10 04:53:48,862 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-10 04:53:48,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-10 04:53:48,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-10 04:53:48,862 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-10 04:53:48,862 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-05-10 04:53:48,862 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-10 04:53:48,862 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-05-10 04:53:48,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-10 04:53:48,862 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,36665,1683694419452' in zk 2023-05-10 04:53:48,862 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-05-10 04:53:48,862 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-05-10 04:53:48,862 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:48,863 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-10 04:53:48,862 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-05-10 04:53:48,862 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,37823,1683694419534' in zk 2023-05-10 04:53:48,863 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-05-10 04:53:48,863 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,37995,1683694419499' in zk 2023-05-10 04:53:48,864 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-05-10 04:53:48,864 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:53:48,864 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-05-10 04:53:48,868 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:48,868 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-05-10 04:53:48,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:48,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:53:48,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:53:48,868 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-05-10 04:53:48,868 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:53:48,868 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:53:48,869 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-05-10 04:53:48,869 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-05-10 04:53:48,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:53:48,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:53:48,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-10 04:53:48,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:48,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:48,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:48,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:53:48,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-10 04:53:48,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:48,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:48,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:48,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,37823,1683694419534': 2023-05-10 04:53:48,872 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37823,1683694419534' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-05-10 04:53:48,872 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-05-10 04:53:48,872 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-10 04:53:48,872 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-10 04:53:48,872 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-05-10 04:53:48,872 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-10 04:53:48,873 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-10 04:53:48,873 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-10 04:53:48,874 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-10 04:53:48,874 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-10 04:53:48,874 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:53:48,873 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-10 04:53:48,873 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-10 04:53:48,874 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-10 04:53:48,874 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:53:48,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-10 04:53:48,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-10 04:53:48,874 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:53:48,874 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:48,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-10 04:53:48,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-10 04:53:48,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:53:48,874 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:53:48,874 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:53:48,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:53:48,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:53:48,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:53:48,875 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:48,875 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:53:48,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:53:48,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-10 04:53:48,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-10 04:53:48,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:53:48,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-10 04:53:48,875 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:48,875 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-10 04:53:48,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:53:48,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-10 04:53:48,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:48,876 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:48,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:48,877 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:48,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:53:48,877 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:48,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-10 04:53:48,878 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:48,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:48,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:48,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:48,880 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:48,880 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:53:48,880 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-10 04:53:48,880 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:53:48,880 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:53:48,880 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:53:48,881 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:53:48,881 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:53:48,881 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:53:48,880 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:48,880 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:53:48,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:53:48,881 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:48,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:53:48,881 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:53:48,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:53:48,881 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-10 04:53:48,881 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:53:48,881 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:48,881 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-05-10 04:53:48,881 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-10 04:53:48,881 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-10 04:53:48,881 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:48,881 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:48,881 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-10 04:53:48,881 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-10 04:53:48,882 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:53:48,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:53:48,882 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:53:48,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:53:48,883 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-05-10 04:53:48,883 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-10 04:53:48,883 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:53:48,883 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:53:48,983 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-10 04:53:48,984 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-10 04:53:49,004 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-10 04:53:49,006 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-05-10 04:53:49,006 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-05-10 04:53:49,006 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-05-10 04:53:49,006 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:53:49,006 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-05-10 04:53:49,006 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-05-10 04:53:49,007 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-10 04:53:49,007 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-10 04:53:49,008 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:53:49,008 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:53:49,008 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:53:49,008 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:53:49,008 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:53:49,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:53:49,008 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:49,008 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:53:49,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:53:49,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:53:49,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-10 04:53:49,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-10 04:53:49,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-10 04:53:49,009 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:49,009 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-10 04:53:49,010 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-10 04:53:49,010 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-10 04:53:49,010 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-10 04:53:49,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:53:49,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-10 04:53:49,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:53:49,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:53:49,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-10 04:53:49,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-10 04:53:49,010 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-05-10 04:53:49,010 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-05-10 04:53:49,010 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-05-10 04:53:49,011 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-05-10 04:53:49,011 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-05-10 04:53:49,011 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-05-10 04:53:49,011 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-05-10 04:53:49,011 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-05-10 04:53:49,011 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:53:49,011 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-05-10 04:53:49,011 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-05-10 04:53:49,011 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:53:49,011 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-05-10 04:53:49,011 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-05-10 04:53:49,011 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-05-10 04:53:49,011 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:53:49,011 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-05-10 04:53:49,011 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-05-10 04:53:49,011 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-05-10 04:53:49,011 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-05-10 04:53:49,011 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-10 04:53:49,011 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37995,1683694419499' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-05-10 04:53:49,012 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1683694427650.6c70954228a0bbe97dfdc76861367c66. 2023-05-10 04:53:49,011 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-10 04:53:49,012 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1683694427650.6c70954228a0bbe97dfdc76861367c66. started... 2023-05-10 04:53:49,011 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36665,1683694419452' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-05-10 04:53:49,012 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 6c70954228a0bbe97dfdc76861367c66 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-10 04:53:49,015 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-10 04:53:49,015 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-10 04:53:49,015 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-10 04:53:49,016 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:53:49,018 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-10 04:53:49,018 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:53:49,027 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/.tmp/cf1/79fdb2e1e5fb412d8fa298aca56435df 2023-05-10 04:53:49,041 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/.tmp/cf2/eff43a97e0c74de982adbbc3fbf1de59 2023-05-10 04:53:49,047 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/.tmp/cf1/79fdb2e1e5fb412d8fa298aca56435df as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf1/79fdb2e1e5fb412d8fa298aca56435df 2023-05-10 04:53:49,051 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf1/79fdb2e1e5fb412d8fa298aca56435df, entries=10, sequenceid=58, filesize=5.0 K 2023-05-10 04:53:49,052 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/.tmp/cf2/eff43a97e0c74de982adbbc3fbf1de59 as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf2/eff43a97e0c74de982adbbc3fbf1de59 2023-05-10 04:53:49,057 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf2/eff43a97e0c74de982adbbc3fbf1de59, entries=10, sequenceid=58, filesize=5.0 K 2023-05-10 04:53:49,058 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 6c70954228a0bbe97dfdc76861367c66 in 46ms, sequenceid=58, compaction requested=true 2023-05-10 04:53:49,058 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 6c70954228a0bbe97dfdc76861367c66: 2023-05-10 04:53:49,058 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1683694427650.6c70954228a0bbe97dfdc76861367c66. 2023-05-10 04:53:49,058 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-10 04:53:49,058 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-10 04:53:49,058 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:49,058 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-05-10 04:53:49,058 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37823,1683694419534' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-05-10 04:53:49,060 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-10 04:53:49,060 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:49,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:49,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:53:49,060 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:53:49,060 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-10 04:53:49,060 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:53:49,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:53:49,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:53:49,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-10 04:53:49,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:49,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:49,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:49,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:53:49,063 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37823,1683694419534' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-05-10 04:53:49,063 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@44da72ae[Count = 0] remaining members to acquire global barrier 2023-05-10 04:53:49,063 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-05-10 04:53:49,063 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-10 04:53:49,064 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-10 04:53:49,064 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-10 04:53:49,064 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-10 04:53:49,064 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-10 04:53:49,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-10 04:53:49,064 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-10 04:53:49,065 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-05-10 04:53:49,065 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-05-10 04:53:49,064 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-10 04:53:49,065 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,37995,1683694419499' in zk 2023-05-10 04:53:49,065 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:49,065 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-10 04:53:49,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-10 04:53:49,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-10 04:53:49,065 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-05-10 04:53:49,065 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-05-10 04:53:49,065 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-05-10 04:53:49,065 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-05-10 04:53:49,065 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,37823,1683694419534' in zk 2023-05-10 04:53:49,065 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,36665,1683694419452' in zk 2023-05-10 04:53:49,068 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-05-10 04:53:49,068 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:53:49,068 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-05-10 04:53:49,069 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:49,069 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-05-10 04:53:49,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:49,069 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:53:49,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:53:49,069 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-05-10 04:53:49,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:53:49,069 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-05-10 04:53:49,069 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:53:49,069 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-05-10 04:53:49,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:53:49,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:53:49,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-10 04:53:49,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:49,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:49,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:49,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:53:49,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-10 04:53:49,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:49,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:49,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:49,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,37823,1683694419534': 2023-05-10 04:53:49,072 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37823,1683694419534' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-05-10 04:53:49,072 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-05-10 04:53:49,072 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-10 04:53:49,072 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-10 04:53:49,072 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-05-10 04:53:49,072 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-10 04:53:49,074 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-10 04:53:49,074 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-10 04:53:49,074 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-10 04:53:49,074 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-10 04:53:49,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-10 04:53:49,074 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-10 04:53:49,074 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:53:49,074 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:53:49,074 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-10 04:53:49,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-10 04:53:49,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:53:49,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:53:49,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-10 04:53:49,074 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:53:49,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:53:49,074 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-10 04:53:49,074 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:53:49,075 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:49,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:53:49,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-10 04:53:49,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-10 04:53:49,075 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:53:49,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:53:49,075 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:49,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-10 04:53:49,075 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:53:49,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:53:49,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-10 04:53:49,075 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:49,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-10 04:53:49,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:53:49,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-10 04:53:49,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:49,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:49,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:49,077 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:49,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:53:49,077 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:49,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-10 04:53:49,077 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:49,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:49,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:49,078 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:49,084 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:49,084 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:53:49,084 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:53:49,084 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:53:49,084 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-10 04:53:49,084 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-05-10 04:53:49,084 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-10 04:53:49,084 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:53:49,084 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:49,084 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:53:49,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:53:49,084 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:53:49,084 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:53:49,084 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:53:49,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:53:49,084 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:49,085 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-10 04:53:49,085 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:49,085 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-10 04:53:49,085 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:49,085 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:49,085 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-10 04:53:49,085 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-10 04:53:49,085 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-05-10 04:53:49,085 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-10 04:53:49,085 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:53:49,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:53:49,084 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:53:49,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:53:49,084 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:53:49,085 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:53:49,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:53:49,086 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:53:49,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:53:49,185 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-10 04:53:49,187 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-10 04:53:49,204 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-10 04:53:49,206 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-05-10 04:53:49,206 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-05-10 04:53:49,206 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-05-10 04:53:49,206 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:53:49,206 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-05-10 04:53:49,206 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-05-10 04:53:49,206 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-10 04:53:49,207 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-10 04:53:49,208 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:49,208 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:53:49,208 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:53:49,208 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:53:49,208 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:53:49,208 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:53:49,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:53:49,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:53:49,208 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:53:49,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:53:49,208 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:49,208 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-10 04:53:49,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-10 04:53:49,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-10 04:53:49,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-10 04:53:49,209 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-10 04:53:49,209 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-10 04:53:49,209 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-10 04:53:49,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:53:49,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-10 04:53:49,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:53:49,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:53:49,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-10 04:53:49,209 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-05-10 04:53:49,210 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-05-10 04:53:49,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-10 04:53:49,210 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-05-10 04:53:49,210 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-05-10 04:53:49,210 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-05-10 04:53:49,210 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-05-10 04:53:49,210 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-05-10 04:53:49,210 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-05-10 04:53:49,210 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-05-10 04:53:49,210 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-05-10 04:53:49,210 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:53:49,210 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-05-10 04:53:49,210 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-05-10 04:53:49,210 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-05-10 04:53:49,210 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:53:49,210 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-10 04:53:49,211 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-10 04:53:49,210 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:53:49,211 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1683694427650.6c70954228a0bbe97dfdc76861367c66. 2023-05-10 04:53:49,211 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-05-10 04:53:49,211 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-05-10 04:53:49,210 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-05-10 04:53:49,211 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-05-10 04:53:49,211 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37995,1683694419499' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-05-10 04:53:49,211 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36665,1683694419452' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-05-10 04:53:49,211 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1683694427650.6c70954228a0bbe97dfdc76861367c66. started... 2023-05-10 04:53:49,211 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 6c70954228a0bbe97dfdc76861367c66 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-10 04:53:49,221 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-10 04:53:49,221 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-10 04:53:49,222 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-10 04:53:49,222 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:53:49,222 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-10 04:53:49,222 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:53:49,229 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/.tmp/cf1/731aba47bdd0433abab5bc5a3bc06822 2023-05-10 04:53:49,244 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/.tmp/cf2/7dc3c674095a4635b6b116c2f1bd84f1 2023-05-10 04:53:49,250 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/.tmp/cf1/731aba47bdd0433abab5bc5a3bc06822 as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf1/731aba47bdd0433abab5bc5a3bc06822 2023-05-10 04:53:49,259 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf1/731aba47bdd0433abab5bc5a3bc06822, entries=10, sequenceid=71, filesize=5.0 K 2023-05-10 04:53:49,260 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/.tmp/cf2/7dc3c674095a4635b6b116c2f1bd84f1 as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf2/7dc3c674095a4635b6b116c2f1bd84f1 2023-05-10 04:53:49,265 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf2/7dc3c674095a4635b6b116c2f1bd84f1, entries=10, sequenceid=71, filesize=5.0 K 2023-05-10 04:53:49,266 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 6c70954228a0bbe97dfdc76861367c66 in 55ms, sequenceid=71, compaction requested=true 2023-05-10 04:53:49,266 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 6c70954228a0bbe97dfdc76861367c66: 2023-05-10 04:53:49,266 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1683694427650.6c70954228a0bbe97dfdc76861367c66. 2023-05-10 04:53:49,266 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-10 04:53:49,266 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-10 04:53:49,266 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:49,266 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-05-10 04:53:49,266 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37823,1683694419534' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-05-10 04:53:49,268 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:49,268 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-10 04:53:49,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:49,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:53:49,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:53:49,268 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-10 04:53:49,268 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:53:49,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:53:49,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:53:49,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-10 04:53:49,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:49,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:49,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:49,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:53:49,270 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37823,1683694419534' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-05-10 04:53:49,270 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@72d8cafc[Count = 0] remaining members to acquire global barrier 2023-05-10 04:53:49,270 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-05-10 04:53:49,270 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-10 04:53:49,273 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-10 04:53:49,273 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-10 04:53:49,273 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-10 04:53:49,273 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-10 04:53:49,273 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-10 04:53:49,273 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-10 04:53:49,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-10 04:53:49,273 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:49,273 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-05-10 04:53:49,273 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-05-10 04:53:49,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-10 04:53:49,273 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,36665,1683694419452' in zk 2023-05-10 04:53:49,273 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-10 04:53:49,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-10 04:53:49,273 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-05-10 04:53:49,274 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-05-10 04:53:49,274 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-05-10 04:53:49,274 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,37995,1683694419499' in zk 2023-05-10 04:53:49,274 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-05-10 04:53:49,274 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase4.apache.org,37823,1683694419534' in zk 2023-05-10 04:53:49,275 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-05-10 04:53:49,275 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:53:49,275 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-05-10 04:53:49,276 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-05-10 04:53:49,276 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-05-10 04:53:49,276 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:53:49,276 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:49,276 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-05-10 04:53:49,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:49,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:53:49,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:53:49,276 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:53:49,276 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-05-10 04:53:49,276 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:53:49,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:53:49,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-10 04:53:49,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:49,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:49,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:49,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:53:49,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-10 04:53:49,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:49,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:49,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:49,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase4.apache.org,37823,1683694419534': 2023-05-10 04:53:49,279 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37823,1683694419534' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-05-10 04:53:49,279 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-05-10 04:53:49,279 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-10 04:53:49,280 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-10 04:53:49,280 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-05-10 04:53:49,280 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-10 04:53:49,281 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-10 04:53:49,281 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-10 04:53:49,281 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-10 04:53:49,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-10 04:53:49,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:53:49,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:53:49,281 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-10 04:53:49,281 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:53:49,281 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-10 04:53:49,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-10 04:53:49,281 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:53:49,281 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-10 04:53:49,282 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:53:49,282 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-10 04:53:49,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-10 04:53:49,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-10 04:53:49,282 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:53:49,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:53:49,282 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:49,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:53:49,282 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:53:49,282 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:53:49,282 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:53:49,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-10 04:53:49,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:53:49,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-10 04:53:49,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-10 04:53:49,283 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:49,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-10 04:53:49,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:53:49,283 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:49,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-10 04:53:49,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:49,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:49,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:49,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:53:49,285 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:49,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-05-10 04:53:49,285 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:49,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:49,285 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:49,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:49,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:49,289 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:49,289 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:53:49,289 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:53:49,289 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:53:49,289 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:53:49,289 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-05-10 04:53:49,290 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:53:49,289 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-10 04:53:49,290 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-10 04:53:49,290 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:49,289 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:53:49,289 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:53:49,289 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:53:49,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:53:49,289 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:53:49,290 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-05-10 04:53:49,289 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:53:49,290 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-10 04:53:49,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:53:49,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:53:49,290 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:49,290 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:53:49,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:53:49,290 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-05-10 04:53:49,291 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:49,291 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-10 04:53:49,291 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:53:49,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:53:49,291 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:49,291 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:53:49,291 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:49,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:53:49,291 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-05-10 04:53:49,291 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-05-10 04:53:49,390 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-10 04:53:49,391 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-10 04:53:49,653 WARN [Listener at localhost/45007] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: true 2023-05-10 04:53:49,655 DEBUG [Listener at localhost/45007] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:49,656 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(1029): Stored pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6c70954228a0bbe97dfdc76861367c66, daughterA=75e2592cb2ac08bd9f617bb9d0a0c896, daughterB=b90fd344aff14f0728a80c3d09539e54 2023-05-10 04:53:49,657 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6c70954228a0bbe97dfdc76861367c66, daughterA=75e2592cb2ac08bd9f617bb9d0a0c896, daughterB=b90fd344aff14f0728a80c3d09539e54 2023-05-10 04:53:49,657 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6c70954228a0bbe97dfdc76861367c66, daughterA=75e2592cb2ac08bd9f617bb9d0a0c896, daughterB=b90fd344aff14f0728a80c3d09539e54 2023-05-10 04:53:49,657 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6c70954228a0bbe97dfdc76861367c66, daughterA=75e2592cb2ac08bd9f617bb9d0a0c896, daughterB=b90fd344aff14f0728a80c3d09539e54 2023-05-10 04:53:49,658 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-10 04:53:49,659 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6c70954228a0bbe97dfdc76861367c66, daughterA=75e2592cb2ac08bd9f617bb9d0a0c896, daughterB=b90fd344aff14f0728a80c3d09539e54 2023-05-10 04:53:49,659 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-10 04:53:49,659 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6c70954228a0bbe97dfdc76861367c66, daughterA=75e2592cb2ac08bd9f617bb9d0a0c896, daughterB=b90fd344aff14f0728a80c3d09539e54 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-10 04:53:49,907 INFO [Listener at localhost/45007] procedure.MasterProcedureTestingUtility(417): Restart 2 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6c70954228a0bbe97dfdc76861367c66, daughterA=75e2592cb2ac08bd9f617bb9d0a0c896, daughterB=b90fd344aff14f0728a80c3d09539e54 2023-05-10 04:53:49,908 INFO [Listener at localhost/45007] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-10 04:53:49,908 INFO [Listener at localhost/45007] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-10 04:53:49,908 INFO [Listener at localhost/45007] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-10 04:53:49,910 ERROR [Listener at localhost/45007] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-1,5,PEWorkerGroup] Thread[HFileArchiver-2,5,PEWorkerGroup] Thread[HFileArchiver-3,5,PEWorkerGroup] Thread[HFileArchiver-4,5,PEWorkerGroup] Thread[HFileArchiver-5,5,PEWorkerGroup] Thread[HFileArchiver-6,5,PEWorkerGroup] 2023-05-10 04:53:49,911 INFO [Listener at localhost/45007] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-10 04:53:49,911 INFO [Listener at localhost/45007] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-10 04:53:49,911 INFO [Listener at localhost/45007] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-10 04:53:49,911 INFO [Listener at localhost/45007] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-10 04:53:49,912 INFO [Listener at localhost/45007] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-10 04:53:49,915 DEBUG [Listener at localhost/45007] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[] 2023-05-10 04:53:49,917 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-10 04:53:49,918 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-10 04:53:49,918 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-10 04:53:49,918 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-10 04:53:49,919 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:53:49,920 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=93c2cc329b1fe1893cfbc28c56960522, daughterA=19040b6538a6bdeed7c833cfb44614b6, daughterB=054f69c06f126aad97b290410d5d6188 2023-05-10 04:53:49,920 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:53:49,921 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:53:49,921 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-10 04:53:49,922 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6c70954228a0bbe97dfdc76861367c66, daughterA=75e2592cb2ac08bd9f617bb9d0a0c896, daughterB=b90fd344aff14f0728a80c3d09539e54 2023-05-10 04:53:49,922 DEBUG [Listener at localhost/45007] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6c70954228a0bbe97dfdc76861367c66, daughterA=75e2592cb2ac08bd9f617bb9d0a0c896, daughterB=b90fd344aff14f0728a80c3d09539e54 held the lock before restarting, call acquireLock to restore it. 2023-05-10 04:53:49,922 INFO [Listener at localhost/45007] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6c70954228a0bbe97dfdc76861367c66, daughterA=75e2592cb2ac08bd9f617bb9d0a0c896, daughterB=b90fd344aff14f0728a80c3d09539e54 2023-05-10 04:53:49,923 INFO [Listener at localhost/45007] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6c70954228a0bbe97dfdc76861367c66, daughterA=75e2592cb2ac08bd9f617bb9d0a0c896, daughterB=b90fd344aff14f0728a80c3d09539e54 2023-05-10 04:53:49,923 INFO [Listener at localhost/45007] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6c70954228a0bbe97dfdc76861367c66, daughterA=75e2592cb2ac08bd9f617bb9d0a0c896, daughterB=b90fd344aff14f0728a80c3d09539e54 2023-05-10 04:53:49,923 INFO [Listener at localhost/45007] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 11 msec 2023-05-10 04:53:49,924 DEBUG [Listener at localhost/45007] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-10 04:53:49,925 DEBUG [Listener at localhost/45007] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,37995,1683694419499, table=hbase:meta, region=1588230740 2023-05-10 04:53:49,926 INFO [Listener at localhost/45007] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683694459925 2023-05-10 04:53:49,927 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6c70954228a0bbe97dfdc76861367c66, daughterA=75e2592cb2ac08bd9f617bb9d0a0c896, daughterB=b90fd344aff14f0728a80c3d09539e54 2023-05-10 04:53:49,927 DEBUG [Listener at localhost/45007] assignment.AssignmentManager(1517): Joining cluster... 2023-05-10 04:53:49,931 DEBUG [Listener at localhost/45007] assignment.RegionStateStore(147): Load hbase:meta entry region=ccfda81dc2f13ffc1429bef67d349028, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36665,1683694419452, regionLocation=jenkins-hbase4.apache.org,36665,1683694419452, openSeqNum=2 2023-05-10 04:53:49,931 DEBUG [Listener at localhost/45007] assignment.RegionStateStore(147): Load hbase:meta entry region=6c70954228a0bbe97dfdc76861367c66, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37823,1683694419534, regionLocation=jenkins-hbase4.apache.org,37823,1683694419534, openSeqNum=2 2023-05-10 04:53:49,931 INFO [Listener at localhost/45007] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-10 04:53:49,931 INFO [Listener at localhost/45007] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683694489931 2023-05-10 04:53:49,932 INFO [Listener at localhost/45007] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683694549932 2023-05-10 04:53:49,932 INFO [Listener at localhost/45007] assignment.AssignmentManager(1536): Joined the cluster in 4 msec 2023-05-10 04:53:49,932 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-10 04:53:49,934 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-10 04:53:49,934 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6c70954228a0bbe97dfdc76861367c66, daughterA=75e2592cb2ac08bd9f617bb9d0a0c896, daughterB=b90fd344aff14f0728a80c3d09539e54 2023-05-10 04:53:49,934 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-10 04:53:49,935 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6c70954228a0bbe97dfdc76861367c66, daughterA=75e2592cb2ac08bd9f617bb9d0a0c896, daughterB=b90fd344aff14f0728a80c3d09539e54 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-10 04:53:50,182 INFO [Listener at localhost/45007] procedure.MasterProcedureTestingUtility(417): Restart 3 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6c70954228a0bbe97dfdc76861367c66, daughterA=75e2592cb2ac08bd9f617bb9d0a0c896, daughterB=b90fd344aff14f0728a80c3d09539e54 2023-05-10 04:53:50,182 INFO [Listener at localhost/45007] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-10 04:53:50,182 INFO [Listener at localhost/45007] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-10 04:53:50,182 INFO [Listener at localhost/45007] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-10 04:53:50,183 ERROR [Listener at localhost/45007] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-10 04:53:50,183 INFO [Listener at localhost/45007] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-10 04:53:50,183 INFO [Listener at localhost/45007] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-10 04:53:50,183 INFO [Listener at localhost/45007] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-10 04:53:50,183 INFO [Listener at localhost/45007] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-10 04:53:50,184 INFO [Listener at localhost/45007] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-10 04:53:50,185 DEBUG [Listener at localhost/45007] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-05-10 04:53:50,186 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-10 04:53:50,186 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-10 04:53:50,186 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-10 04:53:50,187 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-10 04:53:50,187 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:53:50,187 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=93c2cc329b1fe1893cfbc28c56960522, daughterA=19040b6538a6bdeed7c833cfb44614b6, daughterB=054f69c06f126aad97b290410d5d6188 2023-05-10 04:53:50,188 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:53:50,188 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:53:50,188 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-10 04:53:50,188 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6c70954228a0bbe97dfdc76861367c66, daughterA=75e2592cb2ac08bd9f617bb9d0a0c896, daughterB=b90fd344aff14f0728a80c3d09539e54 2023-05-10 04:53:50,189 DEBUG [Listener at localhost/45007] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6c70954228a0bbe97dfdc76861367c66, daughterA=75e2592cb2ac08bd9f617bb9d0a0c896, daughterB=b90fd344aff14f0728a80c3d09539e54 held the lock before restarting, call acquireLock to restore it. 2023-05-10 04:53:50,189 INFO [Listener at localhost/45007] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6c70954228a0bbe97dfdc76861367c66, daughterA=75e2592cb2ac08bd9f617bb9d0a0c896, daughterB=b90fd344aff14f0728a80c3d09539e54 2023-05-10 04:53:50,189 INFO [Listener at localhost/45007] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6c70954228a0bbe97dfdc76861367c66, daughterA=75e2592cb2ac08bd9f617bb9d0a0c896, daughterB=b90fd344aff14f0728a80c3d09539e54 2023-05-10 04:53:50,189 INFO [Listener at localhost/45007] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6c70954228a0bbe97dfdc76861367c66, daughterA=75e2592cb2ac08bd9f617bb9d0a0c896, daughterB=b90fd344aff14f0728a80c3d09539e54 2023-05-10 04:53:50,189 INFO [Listener at localhost/45007] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-05-10 04:53:50,190 DEBUG [Listener at localhost/45007] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-10 04:53:50,190 DEBUG [Listener at localhost/45007] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,37995,1683694419499, table=hbase:meta, region=1588230740 2023-05-10 04:53:50,195 INFO [Listener at localhost/45007] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683694460195 2023-05-10 04:53:50,196 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6c70954228a0bbe97dfdc76861367c66, daughterA=75e2592cb2ac08bd9f617bb9d0a0c896, daughterB=b90fd344aff14f0728a80c3d09539e54 2023-05-10 04:53:50,197 DEBUG [Listener at localhost/45007] assignment.AssignmentManager(1517): Joining cluster... 2023-05-10 04:53:50,199 DEBUG [Listener at localhost/45007] assignment.RegionStateStore(147): Load hbase:meta entry region=ccfda81dc2f13ffc1429bef67d349028, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36665,1683694419452, regionLocation=jenkins-hbase4.apache.org,36665,1683694419452, openSeqNum=2 2023-05-10 04:53:50,200 DEBUG [Listener at localhost/45007] assignment.RegionStateStore(147): Load hbase:meta entry region=6c70954228a0bbe97dfdc76861367c66, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37823,1683694419534, regionLocation=jenkins-hbase4.apache.org,37823,1683694419534, openSeqNum=2 2023-05-10 04:53:50,200 INFO [Listener at localhost/45007] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-10 04:53:50,200 INFO [Listener at localhost/45007] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683694490200 2023-05-10 04:53:50,200 INFO [Listener at localhost/45007] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683694550200 2023-05-10 04:53:50,200 INFO [Listener at localhost/45007] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-05-10 04:53:50,201 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-10 04:53:50,204 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=6c70954228a0bbe97dfdc76861367c66, UNASSIGN}] 2023-05-10 04:53:50,204 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-10 04:53:50,204 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6c70954228a0bbe97dfdc76861367c66, daughterA=75e2592cb2ac08bd9f617bb9d0a0c896, daughterB=b90fd344aff14f0728a80c3d09539e54 2023-05-10 04:53:50,204 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-10 04:53:50,204 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6c70954228a0bbe97dfdc76861367c66, daughterA=75e2592cb2ac08bd9f617bb9d0a0c896, daughterB=b90fd344aff14f0728a80c3d09539e54 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-10 04:53:50,451 INFO [Listener at localhost/45007] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6c70954228a0bbe97dfdc76861367c66, daughterA=75e2592cb2ac08bd9f617bb9d0a0c896, daughterB=b90fd344aff14f0728a80c3d09539e54 2023-05-10 04:53:50,451 INFO [Listener at localhost/45007] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-10 04:53:50,451 INFO [Listener at localhost/45007] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-10 04:53:50,451 INFO [Listener at localhost/45007] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-10 04:53:50,452 ERROR [Listener at localhost/45007] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-10 04:53:50,452 INFO [Listener at localhost/45007] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-10 04:53:50,453 INFO [Listener at localhost/45007] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-10 04:53:50,453 INFO [Listener at localhost/45007] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-10 04:53:50,453 INFO [Listener at localhost/45007] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-10 04:53:50,454 INFO [Listener at localhost/45007] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-10 04:53:50,455 DEBUG [Listener at localhost/45007] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-05-10 04:53:50,456 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-10 04:53:50,456 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-10 04:53:50,457 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-10 04:53:50,457 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-10 04:53:50,458 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:53:50,458 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=93c2cc329b1fe1893cfbc28c56960522, daughterA=19040b6538a6bdeed7c833cfb44614b6, daughterB=054f69c06f126aad97b290410d5d6188 2023-05-10 04:53:50,458 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:53:50,459 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:53:50,459 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-10 04:53:50,459 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6c70954228a0bbe97dfdc76861367c66, daughterA=75e2592cb2ac08bd9f617bb9d0a0c896, daughterB=b90fd344aff14f0728a80c3d09539e54 2023-05-10 04:53:50,459 DEBUG [Listener at localhost/45007] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6c70954228a0bbe97dfdc76861367c66, daughterA=75e2592cb2ac08bd9f617bb9d0a0c896, daughterB=b90fd344aff14f0728a80c3d09539e54 held the lock before restarting, call acquireLock to restore it. 2023-05-10 04:53:50,459 INFO [Listener at localhost/45007] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6c70954228a0bbe97dfdc76861367c66, daughterA=75e2592cb2ac08bd9f617bb9d0a0c896, daughterB=b90fd344aff14f0728a80c3d09539e54 2023-05-10 04:53:50,459 INFO [Listener at localhost/45007] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6c70954228a0bbe97dfdc76861367c66, daughterA=75e2592cb2ac08bd9f617bb9d0a0c896, daughterB=b90fd344aff14f0728a80c3d09539e54 2023-05-10 04:53:50,460 INFO [Listener at localhost/45007] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6c70954228a0bbe97dfdc76861367c66, daughterA=75e2592cb2ac08bd9f617bb9d0a0c896, daughterB=b90fd344aff14f0728a80c3d09539e54 2023-05-10 04:53:50,460 INFO [Listener at localhost/45007] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-05-10 04:53:50,460 DEBUG [Listener at localhost/45007] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-10 04:53:50,461 DEBUG [Listener at localhost/45007] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,37995,1683694419499, table=hbase:meta, region=1588230740 2023-05-10 04:53:50,461 INFO [Listener at localhost/45007] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683694460461 2023-05-10 04:53:50,461 DEBUG [Listener at localhost/45007] assignment.AssignmentManager(1517): Joining cluster... 2023-05-10 04:53:50,463 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6c70954228a0bbe97dfdc76861367c66, daughterA=75e2592cb2ac08bd9f617bb9d0a0c896, daughterB=b90fd344aff14f0728a80c3d09539e54 2023-05-10 04:53:50,466 DEBUG [Listener at localhost/45007] assignment.RegionStateStore(147): Load hbase:meta entry region=ccfda81dc2f13ffc1429bef67d349028, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36665,1683694419452, regionLocation=jenkins-hbase4.apache.org,36665,1683694419452, openSeqNum=2 2023-05-10 04:53:50,466 DEBUG [Listener at localhost/45007] assignment.RegionStateStore(147): Load hbase:meta entry region=6c70954228a0bbe97dfdc76861367c66, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37823,1683694419534, regionLocation=jenkins-hbase4.apache.org,37823,1683694419534, openSeqNum=2 2023-05-10 04:53:50,466 INFO [Listener at localhost/45007] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-10 04:53:50,466 INFO [Listener at localhost/45007] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683694490466 2023-05-10 04:53:50,466 INFO [Listener at localhost/45007] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683694550466 2023-05-10 04:53:50,466 INFO [Listener at localhost/45007] assignment.AssignmentManager(1536): Joined the cluster in 4 msec 2023-05-10 04:53:50,467 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=6c70954228a0bbe97dfdc76861367c66, UNASSIGN}] 2023-05-10 04:53:50,467 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-10 04:53:50,469 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=6c70954228a0bbe97dfdc76861367c66, UNASSIGN 2023-05-10 04:53:50,470 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=6c70954228a0bbe97dfdc76861367c66, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:50,470 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1683694427650.6c70954228a0bbe97dfdc76861367c66.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1683694430470"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683694430470"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683694430470"}]},"ts":"1683694430470"} 2023-05-10 04:53:50,472 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 6c70954228a0bbe97dfdc76861367c66, server=jenkins-hbase4.apache.org,37823,1683694419534}] 2023-05-10 04:53:50,472 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-10 04:53:50,472 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=6c70954228a0bbe97dfdc76861367c66, UNASSIGN 2023-05-10 04:53:50,472 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-10 04:53:50,472 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=6c70954228a0bbe97dfdc76861367c66, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-10 04:53:50,716 INFO [Listener at localhost/45007] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6c70954228a0bbe97dfdc76861367c66, daughterA=75e2592cb2ac08bd9f617bb9d0a0c896, daughterB=b90fd344aff14f0728a80c3d09539e54 2023-05-10 04:53:50,717 INFO [Listener at localhost/45007] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-10 04:53:50,717 INFO [Listener at localhost/45007] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-10 04:53:50,717 INFO [Listener at localhost/45007] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-10 04:53:50,718 ERROR [Listener at localhost/45007] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-10 04:53:50,718 INFO [Listener at localhost/45007] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-10 04:53:50,718 INFO [Listener at localhost/45007] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-10 04:53:50,718 INFO [Listener at localhost/45007] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-10 04:53:50,719 INFO [Listener at localhost/45007] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-10 04:53:50,719 INFO [Listener at localhost/45007] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-10 04:53:50,721 DEBUG [Listener at localhost/45007] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-05-10 04:53:50,721 DEBUG [Listener at localhost/45007] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-05-10 04:53:50,722 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-10 04:53:50,722 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-10 04:53:50,723 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-10 04:53:50,723 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-10 04:53:50,724 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:53:50,724 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=93c2cc329b1fe1893cfbc28c56960522, daughterA=19040b6538a6bdeed7c833cfb44614b6, daughterB=054f69c06f126aad97b290410d5d6188 2023-05-10 04:53:50,724 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:53:50,725 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:53:50,725 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-10 04:53:50,726 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(468): Loading pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6c70954228a0bbe97dfdc76861367c66, daughterA=75e2592cb2ac08bd9f617bb9d0a0c896, daughterB=b90fd344aff14f0728a80c3d09539e54 2023-05-10 04:53:50,727 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=6c70954228a0bbe97dfdc76861367c66, UNASSIGN 2023-05-10 04:53:50,727 DEBUG [Listener at localhost/45007] procedure2.Procedure(966): pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6c70954228a0bbe97dfdc76861367c66, daughterA=75e2592cb2ac08bd9f617bb9d0a0c896, daughterB=b90fd344aff14f0728a80c3d09539e54 held the lock before restarting, call acquireLock to restore it. 2023-05-10 04:53:50,727 INFO [Listener at localhost/45007] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6c70954228a0bbe97dfdc76861367c66, daughterA=75e2592cb2ac08bd9f617bb9d0a0c896, daughterB=b90fd344aff14f0728a80c3d09539e54 2023-05-10 04:53:50,727 INFO [Listener at localhost/45007] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6c70954228a0bbe97dfdc76861367c66, daughterA=75e2592cb2ac08bd9f617bb9d0a0c896, daughterB=b90fd344aff14f0728a80c3d09539e54 2023-05-10 04:53:50,727 INFO [Listener at localhost/45007] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6c70954228a0bbe97dfdc76861367c66, daughterA=75e2592cb2ac08bd9f617bb9d0a0c896, daughterB=b90fd344aff14f0728a80c3d09539e54 2023-05-10 04:53:50,727 DEBUG [Listener at localhost/45007] procedure2.Procedure(966): pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=6c70954228a0bbe97dfdc76861367c66, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-05-10 04:53:50,727 INFO [Listener at localhost/45007] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=6c70954228a0bbe97dfdc76861367c66, UNASSIGN 2023-05-10 04:53:50,728 INFO [Listener at localhost/45007] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-05-10 04:53:50,728 DEBUG [Listener at localhost/45007] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-10 04:53:50,729 DEBUG [Listener at localhost/45007] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,37995,1683694419499, table=hbase:meta, region=1588230740 2023-05-10 04:53:50,729 INFO [Listener at localhost/45007] assignment.AssignmentManager(305): Attach pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=6c70954228a0bbe97dfdc76861367c66, UNASSIGN to state=OFFLINE, location=null, table=testSplitWithoutPONR, region=6c70954228a0bbe97dfdc76861367c66 to restore RIT 2023-05-10 04:53:50,729 INFO [Listener at localhost/45007] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683694460729 2023-05-10 04:53:50,730 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=6c70954228a0bbe97dfdc76861367c66, UNASSIGN 2023-05-10 04:53:50,730 DEBUG [Listener at localhost/45007] assignment.AssignmentManager(1517): Joining cluster... 2023-05-10 04:53:50,733 DEBUG [Listener at localhost/45007] assignment.RegionStateStore(147): Load hbase:meta entry region=ccfda81dc2f13ffc1429bef67d349028, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36665,1683694419452, regionLocation=jenkins-hbase4.apache.org,36665,1683694419452, openSeqNum=2 2023-05-10 04:53:50,733 DEBUG [Listener at localhost/45007] assignment.RegionStateStore(147): Load hbase:meta entry region=6c70954228a0bbe97dfdc76861367c66, regionState=CLOSING, lastHost=jenkins-hbase4.apache.org,37823,1683694419534, regionLocation=jenkins-hbase4.apache.org,37823,1683694419534, openSeqNum=2 2023-05-10 04:53:50,733 INFO [Listener at localhost/45007] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-10 04:53:50,733 INFO [Listener at localhost/45007] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683694490733 2023-05-10 04:53:50,734 INFO [Listener at localhost/45007] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683694550734 2023-05-10 04:53:50,734 INFO [Listener at localhost/45007] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-05-10 04:53:50,734 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=6c70954228a0bbe97dfdc76861367c66, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:50,734 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1683694427650.6c70954228a0bbe97dfdc76861367c66.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1683694430734"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683694430734"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683694430734"}]},"ts":"1683694430734"} 2023-05-10 04:53:50,736 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 6c70954228a0bbe97dfdc76861367c66, server=jenkins-hbase4.apache.org,37823,1683694419534}] 2023-05-10 04:53:50,736 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-10 04:53:50,889 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:50,889 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 6c70954228a0bbe97dfdc76861367c66, disabling compactions & flushes 2023-05-10 04:53:50,890 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitWithoutPONR,,1683694427650.6c70954228a0bbe97dfdc76861367c66. 2023-05-10 04:53:50,890 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitWithoutPONR,,1683694427650.6c70954228a0bbe97dfdc76861367c66. 2023-05-10 04:53:50,890 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitWithoutPONR,,1683694427650.6c70954228a0bbe97dfdc76861367c66. after waiting 0 ms 2023-05-10 04:53:50,890 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitWithoutPONR,,1683694427650.6c70954228a0bbe97dfdc76861367c66. 2023-05-10 04:53:50,890 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 6c70954228a0bbe97dfdc76861367c66 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-05-10 04:53:50,900 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/.tmp/cf1/6708ed14179848199d35a214bbb1223b 2023-05-10 04:53:50,915 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/.tmp/cf2/5dd4de4129d7484893b37527662845b3 2023-05-10 04:53:50,921 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/.tmp/cf1/6708ed14179848199d35a214bbb1223b as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf1/6708ed14179848199d35a214bbb1223b 2023-05-10 04:53:50,926 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf1/6708ed14179848199d35a214bbb1223b, entries=8, sequenceid=82, filesize=5.0 K 2023-05-10 04:53:50,927 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/.tmp/cf2/5dd4de4129d7484893b37527662845b3 as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf2/5dd4de4129d7484893b37527662845b3 2023-05-10 04:53:50,931 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf2/5dd4de4129d7484893b37527662845b3, entries=8, sequenceid=82, filesize=5.0 K 2023-05-10 04:53:50,932 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 6c70954228a0bbe97dfdc76861367c66 in 42ms, sequenceid=82, compaction requested=true 2023-05-10 04:53:50,953 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-05-10 04:53:50,954 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-10 04:53:50,954 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitWithoutPONR,,1683694427650.6c70954228a0bbe97dfdc76861367c66. 2023-05-10 04:53:50,954 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 6c70954228a0bbe97dfdc76861367c66: 2023-05-10 04:53:50,956 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:50,957 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=6c70954228a0bbe97dfdc76861367c66, regionState=CLOSED 2023-05-10 04:53:50,957 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitWithoutPONR,,1683694427650.6c70954228a0bbe97dfdc76861367c66.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1683694430957"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683694430957"}]},"ts":"1683694430957"} 2023-05-10 04:53:50,959 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-10 04:53:50,959 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=30, ppid=29, state=SUCCESS, locked=true; CloseRegionProcedure 6c70954228a0bbe97dfdc76861367c66, server=jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:50,959 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-10 04:53:50,959 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=30, ppid=29, state=SUCCESS, locked=true; CloseRegionProcedure 6c70954228a0bbe97dfdc76861367c66, server=jenkins-hbase4.apache.org,37823,1683694419534 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-10 04:53:50,984 INFO [Listener at localhost/45007] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6c70954228a0bbe97dfdc76861367c66, daughterA=75e2592cb2ac08bd9f617bb9d0a0c896, daughterB=b90fd344aff14f0728a80c3d09539e54 2023-05-10 04:53:50,984 INFO [Listener at localhost/45007] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-10 04:53:50,984 INFO [Listener at localhost/45007] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-10 04:53:50,984 INFO [Listener at localhost/45007] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-10 04:53:50,985 ERROR [Listener at localhost/45007] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-10 04:53:50,985 INFO [Listener at localhost/45007] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-10 04:53:50,985 INFO [Listener at localhost/45007] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-10 04:53:50,986 INFO [Listener at localhost/45007] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-10 04:53:50,986 INFO [Listener at localhost/45007] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-10 04:53:50,987 INFO [Listener at localhost/45007] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-10 04:53:50,988 DEBUG [Listener at localhost/45007] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-05-10 04:53:50,988 DEBUG [Listener at localhost/45007] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-05-10 04:53:50,988 DEBUG [Listener at localhost/45007] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4] 2023-05-10 04:53:50,989 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-10 04:53:50,989 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-10 04:53:50,990 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-10 04:53:50,990 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-10 04:53:50,990 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:53:50,991 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=93c2cc329b1fe1893cfbc28c56960522, daughterA=19040b6538a6bdeed7c833cfb44614b6, daughterB=054f69c06f126aad97b290410d5d6188 2023-05-10 04:53:50,991 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:53:50,991 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:53:50,992 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-10 04:53:50,993 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(468): Loading pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6c70954228a0bbe97dfdc76861367c66, daughterA=75e2592cb2ac08bd9f617bb9d0a0c896, daughterB=b90fd344aff14f0728a80c3d09539e54 2023-05-10 04:53:50,993 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=6c70954228a0bbe97dfdc76861367c66, UNASSIGN 2023-05-10 04:53:50,993 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 6c70954228a0bbe97dfdc76861367c66, server=jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:50,993 DEBUG [Listener at localhost/45007] procedure2.Procedure(966): pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6c70954228a0bbe97dfdc76861367c66, daughterA=75e2592cb2ac08bd9f617bb9d0a0c896, daughterB=b90fd344aff14f0728a80c3d09539e54 held the lock before restarting, call acquireLock to restore it. 2023-05-10 04:53:50,993 INFO [Listener at localhost/45007] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6c70954228a0bbe97dfdc76861367c66, daughterA=75e2592cb2ac08bd9f617bb9d0a0c896, daughterB=b90fd344aff14f0728a80c3d09539e54 2023-05-10 04:53:50,994 INFO [Listener at localhost/45007] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6c70954228a0bbe97dfdc76861367c66, daughterA=75e2592cb2ac08bd9f617bb9d0a0c896, daughterB=b90fd344aff14f0728a80c3d09539e54 2023-05-10 04:53:50,994 INFO [Listener at localhost/45007] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6c70954228a0bbe97dfdc76861367c66, daughterA=75e2592cb2ac08bd9f617bb9d0a0c896, daughterB=b90fd344aff14f0728a80c3d09539e54 2023-05-10 04:53:50,994 DEBUG [Listener at localhost/45007] procedure2.Procedure(966): pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=6c70954228a0bbe97dfdc76861367c66, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-05-10 04:53:50,994 INFO [Listener at localhost/45007] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=6c70954228a0bbe97dfdc76861367c66, UNASSIGN 2023-05-10 04:53:50,994 DEBUG [Listener at localhost/45007] procedure2.Procedure(966): pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 6c70954228a0bbe97dfdc76861367c66, server=jenkins-hbase4.apache.org,37823,1683694419534 held the lock before restarting, call acquireLock to restore it. 2023-05-10 04:53:50,994 INFO [Listener at localhost/45007] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-05-10 04:53:50,995 DEBUG [Listener at localhost/45007] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-10 04:53:50,995 DEBUG [Listener at localhost/45007] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,37995,1683694419499, table=hbase:meta, region=1588230740 2023-05-10 04:53:50,996 INFO [Listener at localhost/45007] assignment.AssignmentManager(305): Attach pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=6c70954228a0bbe97dfdc76861367c66, UNASSIGN to state=OFFLINE, location=null, table=testSplitWithoutPONR, region=6c70954228a0bbe97dfdc76861367c66 to restore RIT 2023-05-10 04:53:50,996 INFO [Listener at localhost/45007] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683694460996 2023-05-10 04:53:50,996 DEBUG [Listener at localhost/45007] assignment.AssignmentManager(1517): Joining cluster... 2023-05-10 04:53:50,997 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 6c70954228a0bbe97dfdc76861367c66, server=jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:50,999 DEBUG [Listener at localhost/45007] assignment.RegionStateStore(147): Load hbase:meta entry region=ccfda81dc2f13ffc1429bef67d349028, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36665,1683694419452, regionLocation=jenkins-hbase4.apache.org,36665,1683694419452, openSeqNum=2 2023-05-10 04:53:50,999 DEBUG [Listener at localhost/45007] assignment.RegionStateStore(147): Load hbase:meta entry region=6c70954228a0bbe97dfdc76861367c66, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,37823,1683694419534, regionLocation=jenkins-hbase4.apache.org,37823,1683694419534, openSeqNum=2 2023-05-10 04:53:51,000 INFO [Listener at localhost/45007] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-10 04:53:51,000 INFO [Listener at localhost/45007] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683694491000 2023-05-10 04:53:51,000 INFO [Listener at localhost/45007] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683694551000 2023-05-10 04:53:51,000 INFO [Listener at localhost/45007] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-05-10 04:53:51,000 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=6c70954228a0bbe97dfdc76861367c66, regionState=CLOSED, regionLocation=jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:51,000 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1683694427650.6c70954228a0bbe97dfdc76861367c66.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1683694431000"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683694431000"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683694431000"}]},"ts":"1683694431000"} 2023-05-10 04:53:51,002 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-10 04:53:51,004 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=30, resume processing ppid=29 2023-05-10 04:53:51,005 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 6c70954228a0bbe97dfdc76861367c66, server=jenkins-hbase4.apache.org,37823,1683694419534 in 266 msec 2023-05-10 04:53:51,005 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-10 04:53:51,005 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=29, ppid=28, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=6c70954228a0bbe97dfdc76861367c66, UNASSIGN 2023-05-10 04:53:51,005 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-10 04:53:51,005 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=29, ppid=28, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=6c70954228a0bbe97dfdc76861367c66, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-10 04:53:51,250 INFO [Listener at localhost/45007] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6c70954228a0bbe97dfdc76861367c66, daughterA=75e2592cb2ac08bd9f617bb9d0a0c896, daughterB=b90fd344aff14f0728a80c3d09539e54 2023-05-10 04:53:51,250 INFO [Listener at localhost/45007] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-10 04:53:51,250 INFO [Listener at localhost/45007] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-10 04:53:51,250 INFO [Listener at localhost/45007] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-10 04:53:51,252 ERROR [Listener at localhost/45007] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-10 04:53:51,252 INFO [Listener at localhost/45007] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-10 04:53:51,252 INFO [Listener at localhost/45007] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-10 04:53:51,252 INFO [Listener at localhost/45007] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-10 04:53:51,252 INFO [Listener at localhost/45007] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-10 04:53:51,253 INFO [Listener at localhost/45007] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-10 04:53:51,255 DEBUG [Listener at localhost/45007] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-05-10 04:53:51,255 DEBUG [Listener at localhost/45007] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-05-10 04:53:51,255 DEBUG [Listener at localhost/45007] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-05-10 04:53:51,255 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-10 04:53:51,256 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-10 04:53:51,256 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-10 04:53:51,257 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-10 04:53:51,257 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:53:51,258 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=93c2cc329b1fe1893cfbc28c56960522, daughterA=19040b6538a6bdeed7c833cfb44614b6, daughterB=054f69c06f126aad97b290410d5d6188 2023-05-10 04:53:51,258 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:53:51,258 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:53:51,259 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-10 04:53:51,260 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(468): Loading pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6c70954228a0bbe97dfdc76861367c66, daughterA=75e2592cb2ac08bd9f617bb9d0a0c896, daughterB=b90fd344aff14f0728a80c3d09539e54 2023-05-10 04:53:51,260 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=6c70954228a0bbe97dfdc76861367c66, UNASSIGN 2023-05-10 04:53:51,260 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 6c70954228a0bbe97dfdc76861367c66, server=jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:51,260 DEBUG [Listener at localhost/45007] procedure2.Procedure(966): pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6c70954228a0bbe97dfdc76861367c66, daughterA=75e2592cb2ac08bd9f617bb9d0a0c896, daughterB=b90fd344aff14f0728a80c3d09539e54 held the lock before restarting, call acquireLock to restore it. 2023-05-10 04:53:51,260 INFO [Listener at localhost/45007] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6c70954228a0bbe97dfdc76861367c66, daughterA=75e2592cb2ac08bd9f617bb9d0a0c896, daughterB=b90fd344aff14f0728a80c3d09539e54 2023-05-10 04:53:51,260 INFO [Listener at localhost/45007] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6c70954228a0bbe97dfdc76861367c66, daughterA=75e2592cb2ac08bd9f617bb9d0a0c896, daughterB=b90fd344aff14f0728a80c3d09539e54 2023-05-10 04:53:51,261 INFO [Listener at localhost/45007] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6c70954228a0bbe97dfdc76861367c66, daughterA=75e2592cb2ac08bd9f617bb9d0a0c896, daughterB=b90fd344aff14f0728a80c3d09539e54 2023-05-10 04:53:51,261 DEBUG [Listener at localhost/45007] procedure2.Procedure(946): pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=6c70954228a0bbe97dfdc76861367c66, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-05-10 04:53:51,261 DEBUG [Listener at localhost/45007] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 6c70954228a0bbe97dfdc76861367c66, server=jenkins-hbase4.apache.org,37823,1683694419534 didn't hold the lock before restarting, skip acquiring lock. 2023-05-10 04:53:51,261 INFO [Listener at localhost/45007] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-05-10 04:53:51,262 DEBUG [Listener at localhost/45007] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-10 04:53:51,262 DEBUG [Listener at localhost/45007] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,37995,1683694419499, table=hbase:meta, region=1588230740 2023-05-10 04:53:51,262 INFO [Listener at localhost/45007] assignment.AssignmentManager(305): Attach pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=6c70954228a0bbe97dfdc76861367c66, UNASSIGN to state=OFFLINE, location=null, table=testSplitWithoutPONR, region=6c70954228a0bbe97dfdc76861367c66 to restore RIT 2023-05-10 04:53:51,263 INFO [Listener at localhost/45007] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683694461263 2023-05-10 04:53:51,263 DEBUG [Listener at localhost/45007] assignment.AssignmentManager(1517): Joining cluster... 2023-05-10 04:53:51,263 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=6c70954228a0bbe97dfdc76861367c66, UNASSIGN 2023-05-10 04:53:51,266 DEBUG [Listener at localhost/45007] assignment.RegionStateStore(147): Load hbase:meta entry region=ccfda81dc2f13ffc1429bef67d349028, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36665,1683694419452, regionLocation=jenkins-hbase4.apache.org,36665,1683694419452, openSeqNum=2 2023-05-10 04:53:51,266 DEBUG [Listener at localhost/45007] assignment.RegionStateStore(147): Load hbase:meta entry region=6c70954228a0bbe97dfdc76861367c66, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,37823,1683694419534, regionLocation=jenkins-hbase4.apache.org,37823,1683694419534, openSeqNum=2 2023-05-10 04:53:51,267 INFO [Listener at localhost/45007] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-10 04:53:51,267 INFO [Listener at localhost/45007] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683694491267 2023-05-10 04:53:51,267 INFO [Listener at localhost/45007] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683694551267 2023-05-10 04:53:51,267 INFO [Listener at localhost/45007] assignment.AssignmentManager(1536): Joined the cluster in 4 msec 2023-05-10 04:53:51,267 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=6c70954228a0bbe97dfdc76861367c66, UNASSIGN 2023-05-10 04:53:51,268 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-10 04:53:51,271 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=29, resume processing ppid=28 2023-05-10 04:53:51,271 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=6c70954228a0bbe97dfdc76861367c66, UNASSIGN in 801 msec 2023-05-10 04:53:51,274 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-10 04:53:51,274 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6c70954228a0bbe97dfdc76861367c66, daughterA=75e2592cb2ac08bd9f617bb9d0a0c896, daughterB=b90fd344aff14f0728a80c3d09539e54 2023-05-10 04:53:51,274 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-10 04:53:51,274 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6c70954228a0bbe97dfdc76861367c66, daughterA=75e2592cb2ac08bd9f617bb9d0a0c896, daughterB=b90fd344aff14f0728a80c3d09539e54 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-10 04:53:51,517 INFO [Listener at localhost/45007] procedure.MasterProcedureTestingUtility(417): Restart 4 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6c70954228a0bbe97dfdc76861367c66, daughterA=75e2592cb2ac08bd9f617bb9d0a0c896, daughterB=b90fd344aff14f0728a80c3d09539e54 2023-05-10 04:53:51,517 INFO [Listener at localhost/45007] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-10 04:53:51,517 INFO [Listener at localhost/45007] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-10 04:53:51,517 INFO [Listener at localhost/45007] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-10 04:53:51,518 ERROR [Listener at localhost/45007] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-10 04:53:51,518 INFO [Listener at localhost/45007] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-10 04:53:51,518 INFO [Listener at localhost/45007] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-10 04:53:51,519 INFO [Listener at localhost/45007] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-10 04:53:51,519 INFO [Listener at localhost/45007] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-10 04:53:51,519 INFO [Listener at localhost/45007] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-10 04:53:51,521 DEBUG [Listener at localhost/45007] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-05-10 04:53:51,521 DEBUG [Listener at localhost/45007] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-05-10 04:53:51,521 DEBUG [Listener at localhost/45007] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-05-10 04:53:51,521 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-10 04:53:51,522 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-10 04:53:51,522 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-10 04:53:51,522 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-10 04:53:51,522 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:53:51,523 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=93c2cc329b1fe1893cfbc28c56960522, daughterA=19040b6538a6bdeed7c833cfb44614b6, daughterB=054f69c06f126aad97b290410d5d6188 2023-05-10 04:53:51,523 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:53:51,523 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:53:51,523 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-10 04:53:51,524 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6c70954228a0bbe97dfdc76861367c66, daughterA=75e2592cb2ac08bd9f617bb9d0a0c896, daughterB=b90fd344aff14f0728a80c3d09539e54 2023-05-10 04:53:51,524 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=6c70954228a0bbe97dfdc76861367c66, UNASSIGN 2023-05-10 04:53:51,524 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 6c70954228a0bbe97dfdc76861367c66, server=jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:51,524 DEBUG [Listener at localhost/45007] procedure2.Procedure(946): pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6c70954228a0bbe97dfdc76861367c66, daughterA=75e2592cb2ac08bd9f617bb9d0a0c896, daughterB=b90fd344aff14f0728a80c3d09539e54 didn't hold the lock before restarting, skip acquiring lock. 2023-05-10 04:53:51,524 DEBUG [Listener at localhost/45007] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=6c70954228a0bbe97dfdc76861367c66, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-05-10 04:53:51,524 DEBUG [Listener at localhost/45007] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 6c70954228a0bbe97dfdc76861367c66, server=jenkins-hbase4.apache.org,37823,1683694419534 didn't hold the lock before restarting, skip acquiring lock. 2023-05-10 04:53:51,524 INFO [Listener at localhost/45007] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-05-10 04:53:51,525 DEBUG [Listener at localhost/45007] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-10 04:53:51,525 DEBUG [Listener at localhost/45007] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,37995,1683694419499, table=hbase:meta, region=1588230740 2023-05-10 04:53:51,526 INFO [Listener at localhost/45007] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683694461526 2023-05-10 04:53:51,527 DEBUG [Listener at localhost/45007] assignment.AssignmentManager(1517): Joining cluster... 2023-05-10 04:53:51,527 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6c70954228a0bbe97dfdc76861367c66, daughterA=75e2592cb2ac08bd9f617bb9d0a0c896, daughterB=b90fd344aff14f0728a80c3d09539e54 2023-05-10 04:53:51,530 DEBUG [Listener at localhost/45007] assignment.RegionStateStore(147): Load hbase:meta entry region=ccfda81dc2f13ffc1429bef67d349028, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36665,1683694419452, regionLocation=jenkins-hbase4.apache.org,36665,1683694419452, openSeqNum=2 2023-05-10 04:53:51,530 DEBUG [Listener at localhost/45007] assignment.RegionStateStore(147): Load hbase:meta entry region=6c70954228a0bbe97dfdc76861367c66, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,37823,1683694419534, regionLocation=jenkins-hbase4.apache.org,37823,1683694419534, openSeqNum=2 2023-05-10 04:53:51,530 INFO [Listener at localhost/45007] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-10 04:53:51,530 INFO [Listener at localhost/45007] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683694491530 2023-05-10 04:53:51,530 INFO [Listener at localhost/45007] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683694551530 2023-05-10 04:53:51,530 INFO [Listener at localhost/45007] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-05-10 04:53:51,530 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6c70954228a0bbe97dfdc76861367c66, daughterA=75e2592cb2ac08bd9f617bb9d0a0c896, daughterB=b90fd344aff14f0728a80c3d09539e54 2023-05-10 04:53:51,530 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6c70954228a0bbe97dfdc76861367c66, daughterA=75e2592cb2ac08bd9f617bb9d0a0c896, daughterB=b90fd344aff14f0728a80c3d09539e54 2023-05-10 04:53:51,531 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6c70954228a0bbe97dfdc76861367c66, daughterA=75e2592cb2ac08bd9f617bb9d0a0c896, daughterB=b90fd344aff14f0728a80c3d09539e54 2023-05-10 04:53:51,536 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-10 04:53:51,545 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=28 splitting 14 storefiles, region=6c70954228a0bbe97dfdc76861367c66, threads=14 2023-05-10 04:53:51,546 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf2/51d8a67670c645628aeb50ddbfc1a8dc for region: 6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:51,546 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf2/5b75cfdc8cfb4ced978b992b5b43f1be for region: 6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:51,546 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf2/5dd4de4129d7484893b37527662845b3 for region: 6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:51,546 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf2/7dc3c674095a4635b6b116c2f1bd84f1 for region: 6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:51,546 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf2/8edaf1b5392e42598afb9d4e334cf824 for region: 6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:51,547 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf2/d5d0539ee2494349beff0d9e436cca78 for region: 6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:51,547 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf2/eff43a97e0c74de982adbbc3fbf1de59 for region: 6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:51,547 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf1/1db12d8725064a0e8efea5e9b8d4ee97 for region: 6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:51,547 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf1/6caf2d6b1504485d93a71a5fd6c7407b for region: 6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:51,547 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf1/692164c792dc453096c76676d73fe8e8 for region: 6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:51,547 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf1/6708ed14179848199d35a214bbb1223b for region: 6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:51,548 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf1/94f3fbf75ed04880bf66a2015ae1796f for region: 6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:51,547 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf1/79fdb2e1e5fb412d8fa298aca56435df for region: 6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:51,547 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf1/731aba47bdd0433abab5bc5a3bc06822 for region: 6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:51,744 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf1/6caf2d6b1504485d93a71a5fd6c7407b for region: 6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:51,751 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf2/5dd4de4129d7484893b37527662845b3 for region: 6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:51,755 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf1/731aba47bdd0433abab5bc5a3bc06822 for region: 6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:51,755 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf2/eff43a97e0c74de982adbbc3fbf1de59 for region: 6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:51,770 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf2/8edaf1b5392e42598afb9d4e334cf824 for region: 6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:51,779 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf1/1db12d8725064a0e8efea5e9b8d4ee97 for region: 6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:51,798 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf2/7dc3c674095a4635b6b116c2f1bd84f1 for region: 6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:51,799 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf1/6708ed14179848199d35a214bbb1223b for region: 6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:51,802 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf2/d5d0539ee2494349beff0d9e436cca78 for region: 6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:51,803 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf1/79fdb2e1e5fb412d8fa298aca56435df for region: 6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:51,805 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf2/5b75cfdc8cfb4ced978b992b5b43f1be for region: 6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:51,816 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf1/692164c792dc453096c76676d73fe8e8 for region: 6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:51,817 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf1/94f3fbf75ed04880bf66a2015ae1796f for region: 6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:52,099 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf2/51d8a67670c645628aeb50ddbfc1a8dc for region: 6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:52,099 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=28 split storefiles for region 6c70954228a0bbe97dfdc76861367c66 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-05-10 04:53:52,139 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-10 04:53:52,139 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6c70954228a0bbe97dfdc76861367c66, daughterA=75e2592cb2ac08bd9f617bb9d0a0c896, daughterB=b90fd344aff14f0728a80c3d09539e54 2023-05-10 04:53:52,139 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-10 04:53:52,139 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6c70954228a0bbe97dfdc76861367c66, daughterA=75e2592cb2ac08bd9f617bb9d0a0c896, daughterB=b90fd344aff14f0728a80c3d09539e54 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-10 04:53:52,281 INFO [Listener at localhost/45007] procedure.MasterProcedureTestingUtility(417): Restart 5 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6c70954228a0bbe97dfdc76861367c66, daughterA=75e2592cb2ac08bd9f617bb9d0a0c896, daughterB=b90fd344aff14f0728a80c3d09539e54 2023-05-10 04:53:52,281 INFO [Listener at localhost/45007] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-10 04:53:52,281 INFO [Listener at localhost/45007] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-10 04:53:52,281 INFO [Listener at localhost/45007] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-10 04:53:52,281 ERROR [Listener at localhost/45007] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-10 04:53:52,281 INFO [Listener at localhost/45007] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-10 04:53:52,281 INFO [Listener at localhost/45007] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-10 04:53:52,281 INFO [Listener at localhost/45007] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-10 04:53:52,281 INFO [Listener at localhost/45007] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-10 04:53:52,282 INFO [Listener at localhost/45007] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-10 04:53:52,283 DEBUG [Listener at localhost/45007] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-05-10 04:53:52,283 DEBUG [Listener at localhost/45007] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-05-10 04:53:52,284 DEBUG [Listener at localhost/45007] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-05-10 04:53:52,284 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-10 04:53:52,284 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-10 04:53:52,285 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-10 04:53:52,285 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-10 04:53:52,286 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:53:52,286 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=93c2cc329b1fe1893cfbc28c56960522, daughterA=19040b6538a6bdeed7c833cfb44614b6, daughterB=054f69c06f126aad97b290410d5d6188 2023-05-10 04:53:52,286 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:53:52,287 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:53:52,287 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-10 04:53:52,288 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6c70954228a0bbe97dfdc76861367c66, daughterA=75e2592cb2ac08bd9f617bb9d0a0c896, daughterB=b90fd344aff14f0728a80c3d09539e54 2023-05-10 04:53:52,288 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=6c70954228a0bbe97dfdc76861367c66, UNASSIGN 2023-05-10 04:53:52,288 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 6c70954228a0bbe97dfdc76861367c66, server=jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:52,288 DEBUG [Listener at localhost/45007] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6c70954228a0bbe97dfdc76861367c66, daughterA=75e2592cb2ac08bd9f617bb9d0a0c896, daughterB=b90fd344aff14f0728a80c3d09539e54 held the lock before restarting, call acquireLock to restore it. 2023-05-10 04:53:52,288 INFO [Listener at localhost/45007] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6c70954228a0bbe97dfdc76861367c66, daughterA=75e2592cb2ac08bd9f617bb9d0a0c896, daughterB=b90fd344aff14f0728a80c3d09539e54 2023-05-10 04:53:52,288 INFO [Listener at localhost/45007] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6c70954228a0bbe97dfdc76861367c66, daughterA=75e2592cb2ac08bd9f617bb9d0a0c896, daughterB=b90fd344aff14f0728a80c3d09539e54 2023-05-10 04:53:52,288 INFO [Listener at localhost/45007] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6c70954228a0bbe97dfdc76861367c66, daughterA=75e2592cb2ac08bd9f617bb9d0a0c896, daughterB=b90fd344aff14f0728a80c3d09539e54 2023-05-10 04:53:52,288 DEBUG [Listener at localhost/45007] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=6c70954228a0bbe97dfdc76861367c66, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-05-10 04:53:52,288 DEBUG [Listener at localhost/45007] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 6c70954228a0bbe97dfdc76861367c66, server=jenkins-hbase4.apache.org,37823,1683694419534 didn't hold the lock before restarting, skip acquiring lock. 2023-05-10 04:53:52,288 INFO [Listener at localhost/45007] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-05-10 04:53:52,289 DEBUG [Listener at localhost/45007] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-10 04:53:52,290 DEBUG [Listener at localhost/45007] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,37995,1683694419499, table=hbase:meta, region=1588230740 2023-05-10 04:53:52,290 INFO [Listener at localhost/45007] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683694462290 2023-05-10 04:53:52,291 DEBUG [Listener at localhost/45007] assignment.AssignmentManager(1517): Joining cluster... 2023-05-10 04:53:52,291 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6c70954228a0bbe97dfdc76861367c66, daughterA=75e2592cb2ac08bd9f617bb9d0a0c896, daughterB=b90fd344aff14f0728a80c3d09539e54 2023-05-10 04:53:52,294 DEBUG [Listener at localhost/45007] assignment.RegionStateStore(147): Load hbase:meta entry region=ccfda81dc2f13ffc1429bef67d349028, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36665,1683694419452, regionLocation=jenkins-hbase4.apache.org,36665,1683694419452, openSeqNum=2 2023-05-10 04:53:52,294 DEBUG [Listener at localhost/45007] assignment.RegionStateStore(147): Load hbase:meta entry region=6c70954228a0bbe97dfdc76861367c66, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,37823,1683694419534, regionLocation=jenkins-hbase4.apache.org,37823,1683694419534, openSeqNum=2 2023-05-10 04:53:52,295 INFO [Listener at localhost/45007] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-10 04:53:52,295 INFO [Listener at localhost/45007] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683694492295 2023-05-10 04:53:52,295 INFO [Listener at localhost/45007] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683694552295 2023-05-10 04:53:52,295 INFO [Listener at localhost/45007] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-05-10 04:53:52,305 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=28 splitting 14 storefiles, region=6c70954228a0bbe97dfdc76861367c66, threads=14 2023-05-10 04:53:52,306 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf2/51d8a67670c645628aeb50ddbfc1a8dc for region: 6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:52,306 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf2/5b75cfdc8cfb4ced978b992b5b43f1be for region: 6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:52,306 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf2/5dd4de4129d7484893b37527662845b3 for region: 6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:52,307 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf2/7dc3c674095a4635b6b116c2f1bd84f1 for region: 6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:52,307 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf2/8edaf1b5392e42598afb9d4e334cf824 for region: 6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:52,307 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf2/d5d0539ee2494349beff0d9e436cca78 for region: 6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:52,307 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf2/eff43a97e0c74de982adbbc3fbf1de59 for region: 6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:52,307 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf1/6708ed14179848199d35a214bbb1223b for region: 6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:52,307 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf1/692164c792dc453096c76676d73fe8e8 for region: 6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:52,307 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf1/1db12d8725064a0e8efea5e9b8d4ee97 for region: 6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:52,308 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf1/6caf2d6b1504485d93a71a5fd6c7407b for region: 6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:52,308 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf1/731aba47bdd0433abab5bc5a3bc06822 for region: 6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:52,308 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf1/79fdb2e1e5fb412d8fa298aca56435df for region: 6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:52,308 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf1/94f3fbf75ed04880bf66a2015ae1796f for region: 6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:52,434 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-05-10 04:53:52,492 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf2/5dd4de4129d7484893b37527662845b3 for region: 6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:52,523 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf2/d5d0539ee2494349beff0d9e436cca78 for region: 6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:52,530 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf2/eff43a97e0c74de982adbbc3fbf1de59 for region: 6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:52,535 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf1/1db12d8725064a0e8efea5e9b8d4ee97 for region: 6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:52,543 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf1/6caf2d6b1504485d93a71a5fd6c7407b for region: 6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:52,543 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf1/94f3fbf75ed04880bf66a2015ae1796f for region: 6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:52,556 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf1/79fdb2e1e5fb412d8fa298aca56435df for region: 6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:52,571 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf1/692164c792dc453096c76676d73fe8e8 for region: 6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:52,587 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf1/731aba47bdd0433abab5bc5a3bc06822 for region: 6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:52,603 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf2/8edaf1b5392e42598afb9d4e334cf824 for region: 6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:52,603 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf2/7dc3c674095a4635b6b116c2f1bd84f1 for region: 6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:52,603 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf2/5b75cfdc8cfb4ced978b992b5b43f1be for region: 6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:52,603 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf2/51d8a67670c645628aeb50ddbfc1a8dc for region: 6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:52,607 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf1/6708ed14179848199d35a214bbb1223b for region: 6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:52,607 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=28 split storefiles for region 6c70954228a0bbe97dfdc76861367c66 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-05-10 04:53:52,651 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-10 04:53:52,657 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-05-10 04:53:52,659 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-05-10 04:53:52,659 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-10 04:53:52,659 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6c70954228a0bbe97dfdc76861367c66, daughterA=75e2592cb2ac08bd9f617bb9d0a0c896, daughterB=b90fd344aff14f0728a80c3d09539e54 2023-05-10 04:53:52,659 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-10 04:53:52,659 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6c70954228a0bbe97dfdc76861367c66, daughterA=75e2592cb2ac08bd9f617bb9d0a0c896, daughterB=b90fd344aff14f0728a80c3d09539e54 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-10 04:53:52,795 INFO [Listener at localhost/45007] procedure.MasterProcedureTestingUtility(417): Restart 6 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6c70954228a0bbe97dfdc76861367c66, daughterA=75e2592cb2ac08bd9f617bb9d0a0c896, daughterB=b90fd344aff14f0728a80c3d09539e54 2023-05-10 04:53:52,795 INFO [Listener at localhost/45007] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-10 04:53:52,795 INFO [Listener at localhost/45007] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-10 04:53:52,795 INFO [Listener at localhost/45007] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-10 04:53:52,796 ERROR [Listener at localhost/45007] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-10 04:53:52,796 INFO [Listener at localhost/45007] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-10 04:53:52,796 INFO [Listener at localhost/45007] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-10 04:53:52,796 INFO [Listener at localhost/45007] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-10 04:53:52,796 INFO [Listener at localhost/45007] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-10 04:53:52,797 INFO [Listener at localhost/45007] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-10 04:53:52,798 DEBUG [Listener at localhost/45007] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-05-10 04:53:52,798 DEBUG [Listener at localhost/45007] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-05-10 04:53:52,798 DEBUG [Listener at localhost/45007] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-05-10 04:53:52,798 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-10 04:53:52,799 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-10 04:53:52,799 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-10 04:53:52,799 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-10 04:53:52,799 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:53:52,800 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=93c2cc329b1fe1893cfbc28c56960522, daughterA=19040b6538a6bdeed7c833cfb44614b6, daughterB=054f69c06f126aad97b290410d5d6188 2023-05-10 04:53:52,800 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:53:52,800 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:53:52,801 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-10 04:53:52,801 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6c70954228a0bbe97dfdc76861367c66, daughterA=75e2592cb2ac08bd9f617bb9d0a0c896, daughterB=b90fd344aff14f0728a80c3d09539e54 2023-05-10 04:53:52,801 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=6c70954228a0bbe97dfdc76861367c66, UNASSIGN 2023-05-10 04:53:52,801 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 6c70954228a0bbe97dfdc76861367c66, server=jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:52,802 DEBUG [Listener at localhost/45007] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6c70954228a0bbe97dfdc76861367c66, daughterA=75e2592cb2ac08bd9f617bb9d0a0c896, daughterB=b90fd344aff14f0728a80c3d09539e54 held the lock before restarting, call acquireLock to restore it. 2023-05-10 04:53:52,802 INFO [Listener at localhost/45007] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6c70954228a0bbe97dfdc76861367c66, daughterA=75e2592cb2ac08bd9f617bb9d0a0c896, daughterB=b90fd344aff14f0728a80c3d09539e54 2023-05-10 04:53:52,802 INFO [Listener at localhost/45007] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6c70954228a0bbe97dfdc76861367c66, daughterA=75e2592cb2ac08bd9f617bb9d0a0c896, daughterB=b90fd344aff14f0728a80c3d09539e54 2023-05-10 04:53:52,802 INFO [Listener at localhost/45007] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6c70954228a0bbe97dfdc76861367c66, daughterA=75e2592cb2ac08bd9f617bb9d0a0c896, daughterB=b90fd344aff14f0728a80c3d09539e54 2023-05-10 04:53:52,802 DEBUG [Listener at localhost/45007] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=6c70954228a0bbe97dfdc76861367c66, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-05-10 04:53:52,802 DEBUG [Listener at localhost/45007] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 6c70954228a0bbe97dfdc76861367c66, server=jenkins-hbase4.apache.org,37823,1683694419534 didn't hold the lock before restarting, skip acquiring lock. 2023-05-10 04:53:52,802 INFO [Listener at localhost/45007] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-05-10 04:53:52,803 DEBUG [Listener at localhost/45007] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-10 04:53:52,803 DEBUG [Listener at localhost/45007] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,37995,1683694419499, table=hbase:meta, region=1588230740 2023-05-10 04:53:52,803 INFO [Listener at localhost/45007] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683694462803 2023-05-10 04:53:52,805 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6c70954228a0bbe97dfdc76861367c66, daughterA=75e2592cb2ac08bd9f617bb9d0a0c896, daughterB=b90fd344aff14f0728a80c3d09539e54 2023-05-10 04:53:52,805 DEBUG [Listener at localhost/45007] assignment.AssignmentManager(1517): Joining cluster... 2023-05-10 04:53:52,807 DEBUG [Listener at localhost/45007] assignment.RegionStateStore(147): Load hbase:meta entry region=ccfda81dc2f13ffc1429bef67d349028, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36665,1683694419452, regionLocation=jenkins-hbase4.apache.org,36665,1683694419452, openSeqNum=2 2023-05-10 04:53:52,808 DEBUG [Listener at localhost/45007] assignment.RegionStateStore(147): Load hbase:meta entry region=6c70954228a0bbe97dfdc76861367c66, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,37823,1683694419534, regionLocation=jenkins-hbase4.apache.org,37823,1683694419534, openSeqNum=2 2023-05-10 04:53:52,808 INFO [Listener at localhost/45007] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-10 04:53:52,808 INFO [Listener at localhost/45007] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683694492808 2023-05-10 04:53:52,808 INFO [Listener at localhost/45007] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683694552808 2023-05-10 04:53:52,808 INFO [Listener at localhost/45007] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-05-10 04:53:52,811 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-10 04:53:52,813 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-10 04:53:52,813 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6c70954228a0bbe97dfdc76861367c66, daughterA=75e2592cb2ac08bd9f617bb9d0a0c896, daughterB=b90fd344aff14f0728a80c3d09539e54 2023-05-10 04:53:52,813 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-10 04:53:52,813 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6c70954228a0bbe97dfdc76861367c66, daughterA=75e2592cb2ac08bd9f617bb9d0a0c896, daughterB=b90fd344aff14f0728a80c3d09539e54 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-10 04:53:53,058 WARN [Listener at localhost/45007] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-05-10 04:53:53,058 INFO [Listener at localhost/45007] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-10 04:53:53,058 INFO [Listener at localhost/45007] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-10 04:53:53,059 INFO [Listener at localhost/45007] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-10 04:53:53,059 ERROR [Listener at localhost/45007] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-10 04:53:53,059 INFO [Listener at localhost/45007] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-10 04:53:53,059 INFO [Listener at localhost/45007] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-10 04:53:53,059 INFO [Listener at localhost/45007] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-10 04:53:53,059 INFO [Listener at localhost/45007] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-10 04:53:53,060 INFO [Listener at localhost/45007] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-10 04:53:53,061 DEBUG [Listener at localhost/45007] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-05-10 04:53:53,061 DEBUG [Listener at localhost/45007] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-05-10 04:53:53,061 DEBUG [Listener at localhost/45007] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-05-10 04:53:53,061 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-10 04:53:53,061 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-10 04:53:53,062 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-10 04:53:53,062 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-10 04:53:53,062 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:53:53,063 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=93c2cc329b1fe1893cfbc28c56960522, daughterA=19040b6538a6bdeed7c833cfb44614b6, daughterB=054f69c06f126aad97b290410d5d6188 2023-05-10 04:53:53,063 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:53:53,063 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:53:53,063 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-10 04:53:53,064 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6c70954228a0bbe97dfdc76861367c66, daughterA=75e2592cb2ac08bd9f617bb9d0a0c896, daughterB=b90fd344aff14f0728a80c3d09539e54 2023-05-10 04:53:53,064 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=6c70954228a0bbe97dfdc76861367c66, UNASSIGN 2023-05-10 04:53:53,064 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 6c70954228a0bbe97dfdc76861367c66, server=jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:53,064 DEBUG [Listener at localhost/45007] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6c70954228a0bbe97dfdc76861367c66, daughterA=75e2592cb2ac08bd9f617bb9d0a0c896, daughterB=b90fd344aff14f0728a80c3d09539e54 held the lock before restarting, call acquireLock to restore it. 2023-05-10 04:53:53,064 INFO [Listener at localhost/45007] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6c70954228a0bbe97dfdc76861367c66, daughterA=75e2592cb2ac08bd9f617bb9d0a0c896, daughterB=b90fd344aff14f0728a80c3d09539e54 2023-05-10 04:53:53,064 INFO [Listener at localhost/45007] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6c70954228a0bbe97dfdc76861367c66, daughterA=75e2592cb2ac08bd9f617bb9d0a0c896, daughterB=b90fd344aff14f0728a80c3d09539e54 2023-05-10 04:53:53,064 INFO [Listener at localhost/45007] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6c70954228a0bbe97dfdc76861367c66, daughterA=75e2592cb2ac08bd9f617bb9d0a0c896, daughterB=b90fd344aff14f0728a80c3d09539e54 2023-05-10 04:53:53,064 DEBUG [Listener at localhost/45007] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=6c70954228a0bbe97dfdc76861367c66, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-05-10 04:53:53,064 DEBUG [Listener at localhost/45007] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 6c70954228a0bbe97dfdc76861367c66, server=jenkins-hbase4.apache.org,37823,1683694419534 didn't hold the lock before restarting, skip acquiring lock. 2023-05-10 04:53:53,064 INFO [Listener at localhost/45007] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-05-10 04:53:53,065 DEBUG [Listener at localhost/45007] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-10 04:53:53,065 DEBUG [Listener at localhost/45007] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,37995,1683694419499, table=hbase:meta, region=1588230740 2023-05-10 04:53:53,066 INFO [Listener at localhost/45007] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683694463066 2023-05-10 04:53:53,068 DEBUG [Listener at localhost/45007] assignment.AssignmentManager(1517): Joining cluster... 2023-05-10 04:53:53,068 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6c70954228a0bbe97dfdc76861367c66, daughterA=75e2592cb2ac08bd9f617bb9d0a0c896, daughterB=b90fd344aff14f0728a80c3d09539e54 2023-05-10 04:53:53,070 DEBUG [Listener at localhost/45007] assignment.RegionStateStore(147): Load hbase:meta entry region=ccfda81dc2f13ffc1429bef67d349028, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36665,1683694419452, regionLocation=jenkins-hbase4.apache.org,36665,1683694419452, openSeqNum=2 2023-05-10 04:53:53,070 DEBUG [Listener at localhost/45007] assignment.RegionStateStore(147): Load hbase:meta entry region=6c70954228a0bbe97dfdc76861367c66, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,37823,1683694419534, regionLocation=jenkins-hbase4.apache.org,37823,1683694419534, openSeqNum=2 2023-05-10 04:53:53,070 INFO [Listener at localhost/45007] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-10 04:53:53,070 INFO [Listener at localhost/45007] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683694493070 2023-05-10 04:53:53,070 INFO [Listener at localhost/45007] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683694553070 2023-05-10 04:53:53,070 INFO [Listener at localhost/45007] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-05-10 04:53:53,074 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1683694427650.6c70954228a0bbe97dfdc76861367c66.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1683694433074"},{"qualifier":"splitA","vlen":56,"tag":[],"timestamp":"1683694433074"},{"qualifier":"splitB","vlen":56,"tag":[],"timestamp":"1683694433074"}]},"ts":"1683694433074"} 2023-05-10 04:53:53,074 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1683694429655.75e2592cb2ac08bd9f617bb9d0a0c896.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1683694433074"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683694433074"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683694433074"}]},"ts":"1683694433074"} 2023-05-10 04:53:53,074 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitWithoutPONR,41,1683694429655.b90fd344aff14f0728a80c3d09539e54.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1683694433074"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683694433074"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683694433074"}]},"ts":"1683694433074"} 2023-05-10 04:53:53,080 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=31, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=75e2592cb2ac08bd9f617bb9d0a0c896, ASSIGN}, {pid=32, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=b90fd344aff14f0728a80c3d09539e54, ASSIGN}] 2023-05-10 04:53:53,081 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=32, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=b90fd344aff14f0728a80c3d09539e54, ASSIGN 2023-05-10 04:53:53,082 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=32, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=b90fd344aff14f0728a80c3d09539e54, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,37823,1683694419534; forceNewPlan=false, retain=false 2023-05-10 04:53:53,083 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=31, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=75e2592cb2ac08bd9f617bb9d0a0c896, ASSIGN 2023-05-10 04:53:53,084 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=31, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=75e2592cb2ac08bd9f617bb9d0a0c896, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,37823,1683694419534; forceNewPlan=false, retain=false 2023-05-10 04:53:53,232 INFO [jenkins-hbase4:37903] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-05-10 04:53:53,234 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=31 updating hbase:meta row=75e2592cb2ac08bd9f617bb9d0a0c896, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:53,234 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1683694429655.75e2592cb2ac08bd9f617bb9d0a0c896.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1683694433234"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683694433234"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683694433234"}]},"ts":"1683694433234"} 2023-05-10 04:53:53,236 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=33, ppid=31, state=RUNNABLE; OpenRegionProcedure 75e2592cb2ac08bd9f617bb9d0a0c896, server=jenkins-hbase4.apache.org,37823,1683694419534}] 2023-05-10 04:53:53,240 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=32 updating hbase:meta row=b90fd344aff14f0728a80c3d09539e54, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:53,240 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,41,1683694429655.b90fd344aff14f0728a80c3d09539e54.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1683694433240"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683694433240"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683694433240"}]},"ts":"1683694433240"} 2023-05-10 04:53:53,241 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=34, ppid=32, state=RUNNABLE; OpenRegionProcedure b90fd344aff14f0728a80c3d09539e54, server=jenkins-hbase4.apache.org,37823,1683694419534}] 2023-05-10 04:53:53,392 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitWithoutPONR,41,1683694429655.b90fd344aff14f0728a80c3d09539e54. 2023-05-10 04:53:53,393 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => b90fd344aff14f0728a80c3d09539e54, NAME => 'testSplitWithoutPONR,41,1683694429655.b90fd344aff14f0728a80c3d09539e54.', STARTKEY => '41', ENDKEY => ''} 2023-05-10 04:53:53,393 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-05-10 04:53:53,393 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitWithoutPONR b90fd344aff14f0728a80c3d09539e54 2023-05-10 04:53:53,393 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitWithoutPONR,41,1683694429655.b90fd344aff14f0728a80c3d09539e54.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-10 04:53:53,393 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for b90fd344aff14f0728a80c3d09539e54 2023-05-10 04:53:53,393 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for b90fd344aff14f0728a80c3d09539e54 2023-05-10 04:53:53,395 INFO [StoreOpener-b90fd344aff14f0728a80c3d09539e54-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region b90fd344aff14f0728a80c3d09539e54 2023-05-10 04:53:53,395 DEBUG [StoreOpener-b90fd344aff14f0728a80c3d09539e54-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/cf1 2023-05-10 04:53:53,396 DEBUG [StoreOpener-b90fd344aff14f0728a80c3d09539e54-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/cf1 2023-05-10 04:53:53,396 INFO [StoreOpener-b90fd344aff14f0728a80c3d09539e54-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b90fd344aff14f0728a80c3d09539e54 columnFamilyName cf1 2023-05-10 04:53:53,412 DEBUG [StoreOpener-b90fd344aff14f0728a80c3d09539e54-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/cf1/1db12d8725064a0e8efea5e9b8d4ee97.6c70954228a0bbe97dfdc76861367c66->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf1/1db12d8725064a0e8efea5e9b8d4ee97-top 2023-05-10 04:53:53,417 DEBUG [StoreOpener-b90fd344aff14f0728a80c3d09539e54-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/cf1/6708ed14179848199d35a214bbb1223b.6c70954228a0bbe97dfdc76861367c66->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf1/6708ed14179848199d35a214bbb1223b-top 2023-05-10 04:53:53,422 DEBUG [StoreOpener-b90fd344aff14f0728a80c3d09539e54-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/cf1/692164c792dc453096c76676d73fe8e8.6c70954228a0bbe97dfdc76861367c66->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf1/692164c792dc453096c76676d73fe8e8-top 2023-05-10 04:53:53,427 DEBUG [StoreOpener-b90fd344aff14f0728a80c3d09539e54-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/cf1/6caf2d6b1504485d93a71a5fd6c7407b.6c70954228a0bbe97dfdc76861367c66->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf1/6caf2d6b1504485d93a71a5fd6c7407b-top 2023-05-10 04:53:53,431 DEBUG [StoreOpener-b90fd344aff14f0728a80c3d09539e54-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/cf1/731aba47bdd0433abab5bc5a3bc06822.6c70954228a0bbe97dfdc76861367c66->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf1/731aba47bdd0433abab5bc5a3bc06822-top 2023-05-10 04:53:53,436 DEBUG [StoreOpener-b90fd344aff14f0728a80c3d09539e54-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/cf1/79fdb2e1e5fb412d8fa298aca56435df.6c70954228a0bbe97dfdc76861367c66->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf1/79fdb2e1e5fb412d8fa298aca56435df-top 2023-05-10 04:53:53,441 DEBUG [StoreOpener-b90fd344aff14f0728a80c3d09539e54-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/cf1/94f3fbf75ed04880bf66a2015ae1796f.6c70954228a0bbe97dfdc76861367c66->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf1/94f3fbf75ed04880bf66a2015ae1796f-top 2023-05-10 04:53:53,441 INFO [StoreOpener-b90fd344aff14f0728a80c3d09539e54-1] regionserver.HStore(310): Store=b90fd344aff14f0728a80c3d09539e54/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-10 04:53:53,441 INFO [StoreOpener-b90fd344aff14f0728a80c3d09539e54-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region b90fd344aff14f0728a80c3d09539e54 2023-05-10 04:53:53,442 DEBUG [StoreOpener-b90fd344aff14f0728a80c3d09539e54-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/cf2 2023-05-10 04:53:53,442 DEBUG [StoreOpener-b90fd344aff14f0728a80c3d09539e54-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/cf2 2023-05-10 04:53:53,443 INFO [StoreOpener-b90fd344aff14f0728a80c3d09539e54-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region b90fd344aff14f0728a80c3d09539e54 columnFamilyName cf2 2023-05-10 04:53:53,459 DEBUG [StoreOpener-b90fd344aff14f0728a80c3d09539e54-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/cf2/51d8a67670c645628aeb50ddbfc1a8dc.6c70954228a0bbe97dfdc76861367c66->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf2/51d8a67670c645628aeb50ddbfc1a8dc-top 2023-05-10 04:53:53,464 DEBUG [StoreOpener-b90fd344aff14f0728a80c3d09539e54-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/cf2/5b75cfdc8cfb4ced978b992b5b43f1be.6c70954228a0bbe97dfdc76861367c66->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf2/5b75cfdc8cfb4ced978b992b5b43f1be-top 2023-05-10 04:53:53,468 DEBUG [StoreOpener-b90fd344aff14f0728a80c3d09539e54-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/cf2/5dd4de4129d7484893b37527662845b3.6c70954228a0bbe97dfdc76861367c66->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf2/5dd4de4129d7484893b37527662845b3-top 2023-05-10 04:53:53,473 DEBUG [StoreOpener-b90fd344aff14f0728a80c3d09539e54-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/cf2/7dc3c674095a4635b6b116c2f1bd84f1.6c70954228a0bbe97dfdc76861367c66->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf2/7dc3c674095a4635b6b116c2f1bd84f1-top 2023-05-10 04:53:53,478 DEBUG [StoreOpener-b90fd344aff14f0728a80c3d09539e54-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/cf2/8edaf1b5392e42598afb9d4e334cf824.6c70954228a0bbe97dfdc76861367c66->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf2/8edaf1b5392e42598afb9d4e334cf824-top 2023-05-10 04:53:53,483 DEBUG [StoreOpener-b90fd344aff14f0728a80c3d09539e54-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/cf2/d5d0539ee2494349beff0d9e436cca78.6c70954228a0bbe97dfdc76861367c66->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf2/d5d0539ee2494349beff0d9e436cca78-top 2023-05-10 04:53:53,487 DEBUG [StoreOpener-b90fd344aff14f0728a80c3d09539e54-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/cf2/eff43a97e0c74de982adbbc3fbf1de59.6c70954228a0bbe97dfdc76861367c66->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf2/eff43a97e0c74de982adbbc3fbf1de59-top 2023-05-10 04:53:53,487 INFO [StoreOpener-b90fd344aff14f0728a80c3d09539e54-1] regionserver.HStore(310): Store=b90fd344aff14f0728a80c3d09539e54/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-10 04:53:53,488 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54 2023-05-10 04:53:53,489 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54 2023-05-10 04:53:53,491 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitWithoutPONR descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-05-10 04:53:53,493 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for b90fd344aff14f0728a80c3d09539e54 2023-05-10 04:53:53,494 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened b90fd344aff14f0728a80c3d09539e54; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65353746, jitterRate=-0.02615329623222351}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-05-10 04:53:53,494 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for b90fd344aff14f0728a80c3d09539e54: 2023-05-10 04:53:53,495 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2334): Post open deploy tasks for testSplitWithoutPONR,41,1683694429655.b90fd344aff14f0728a80c3d09539e54., pid=34, masterSystemTime=1683694433389 2023-05-10 04:53:53,495 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-10 04:53:53,496 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-05-10 04:53:53,496 DEBUG [RS:2;jenkins-hbase4:37823-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-05-10 04:53:53,497 DEBUG [RS:2;jenkins-hbase4:37823-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-05-10 04:53:53,497 INFO [RS:2;jenkins-hbase4:37823-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitWithoutPONR,41,1683694429655.b90fd344aff14f0728a80c3d09539e54. 2023-05-10 04:53:53,497 INFO [RS:2;jenkins-hbase4:37823-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitWithoutPONR,41,1683694429655.b90fd344aff14f0728a80c3d09539e54. 2023-05-10 04:53:53,497 DEBUG [RS:2;jenkins-hbase4:37823-shortCompactions-0] regionserver.HStore(1912): b90fd344aff14f0728a80c3d09539e54/cf1 is initiating minor compaction (all files) 2023-05-10 04:53:53,497 DEBUG [RS:2;jenkins-hbase4:37823-longCompactions-0] regionserver.HStore(1912): b90fd344aff14f0728a80c3d09539e54/cf2 is initiating minor compaction (all files) 2023-05-10 04:53:53,497 INFO [RS:2;jenkins-hbase4:37823-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of b90fd344aff14f0728a80c3d09539e54/cf1 in testSplitWithoutPONR,41,1683694429655.b90fd344aff14f0728a80c3d09539e54. 2023-05-10 04:53:53,497 INFO [RS:2;jenkins-hbase4:37823-longCompactions-0] regionserver.HRegion(2259): Starting compaction of b90fd344aff14f0728a80c3d09539e54/cf2 in testSplitWithoutPONR,41,1683694429655.b90fd344aff14f0728a80c3d09539e54. 2023-05-10 04:53:53,498 INFO [RS:2;jenkins-hbase4:37823-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/cf2/5b75cfdc8cfb4ced978b992b5b43f1be.6c70954228a0bbe97dfdc76861367c66->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf2/5b75cfdc8cfb4ced978b992b5b43f1be-top, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/cf2/51d8a67670c645628aeb50ddbfc1a8dc.6c70954228a0bbe97dfdc76861367c66->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf2/51d8a67670c645628aeb50ddbfc1a8dc-top, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/cf2/8edaf1b5392e42598afb9d4e334cf824.6c70954228a0bbe97dfdc76861367c66->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf2/8edaf1b5392e42598afb9d4e334cf824-top, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/cf2/d5d0539ee2494349beff0d9e436cca78.6c70954228a0bbe97dfdc76861367c66->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf2/d5d0539ee2494349beff0d9e436cca78-top, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/cf2/eff43a97e0c74de982adbbc3fbf1de59.6c70954228a0bbe97dfdc76861367c66->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf2/eff43a97e0c74de982adbbc3fbf1de59-top, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/cf2/7dc3c674095a4635b6b116c2f1bd84f1.6c70954228a0bbe97dfdc76861367c66->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf2/7dc3c674095a4635b6b116c2f1bd84f1-top, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/cf2/5dd4de4129d7484893b37527662845b3.6c70954228a0bbe97dfdc76861367c66->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf2/5dd4de4129d7484893b37527662845b3-top] into tmpdir=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/.tmp, totalSize=34.9 K 2023-05-10 04:53:53,498 INFO [RS:2;jenkins-hbase4:37823-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/cf1/692164c792dc453096c76676d73fe8e8.6c70954228a0bbe97dfdc76861367c66->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf1/692164c792dc453096c76676d73fe8e8-top, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/cf1/94f3fbf75ed04880bf66a2015ae1796f.6c70954228a0bbe97dfdc76861367c66->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf1/94f3fbf75ed04880bf66a2015ae1796f-top, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/cf1/1db12d8725064a0e8efea5e9b8d4ee97.6c70954228a0bbe97dfdc76861367c66->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf1/1db12d8725064a0e8efea5e9b8d4ee97-top, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/cf1/6caf2d6b1504485d93a71a5fd6c7407b.6c70954228a0bbe97dfdc76861367c66->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf1/6caf2d6b1504485d93a71a5fd6c7407b-top, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/cf1/79fdb2e1e5fb412d8fa298aca56435df.6c70954228a0bbe97dfdc76861367c66->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf1/79fdb2e1e5fb412d8fa298aca56435df-top, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/cf1/731aba47bdd0433abab5bc5a3bc06822.6c70954228a0bbe97dfdc76861367c66->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf1/731aba47bdd0433abab5bc5a3bc06822-top, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/cf1/6708ed14179848199d35a214bbb1223b.6c70954228a0bbe97dfdc76861367c66->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf1/6708ed14179848199d35a214bbb1223b-top] into tmpdir=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/.tmp, totalSize=34.9 K 2023-05-10 04:53:53,499 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2361): Finished post open deploy task for testSplitWithoutPONR,41,1683694429655.b90fd344aff14f0728a80c3d09539e54. 2023-05-10 04:53:53,499 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitWithoutPONR,41,1683694429655.b90fd344aff14f0728a80c3d09539e54. 2023-05-10 04:53:53,499 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitWithoutPONR,,1683694429655.75e2592cb2ac08bd9f617bb9d0a0c896. 2023-05-10 04:53:53,499 DEBUG [RS:2;jenkins-hbase4:37823-longCompactions-0] compactions.Compactor(207): Compacting 5b75cfdc8cfb4ced978b992b5b43f1be.6c70954228a0bbe97dfdc76861367c66, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1683694428155 2023-05-10 04:53:53,499 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 75e2592cb2ac08bd9f617bb9d0a0c896, NAME => 'testSplitWithoutPONR,,1683694429655.75e2592cb2ac08bd9f617bb9d0a0c896.', STARTKEY => '', ENDKEY => '41'} 2023-05-10 04:53:53,499 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-05-10 04:53:53,500 DEBUG [RS:2;jenkins-hbase4:37823-longCompactions-0] compactions.Compactor(207): Compacting 51d8a67670c645628aeb50ddbfc1a8dc.6c70954228a0bbe97dfdc76861367c66, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1683694428364 2023-05-10 04:53:53,500 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=32 updating hbase:meta row=b90fd344aff14f0728a80c3d09539e54, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:53,499 DEBUG [RS:2;jenkins-hbase4:37823-shortCompactions-0] compactions.Compactor(207): Compacting 692164c792dc453096c76676d73fe8e8.6c70954228a0bbe97dfdc76861367c66, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1683694428155 2023-05-10 04:53:53,500 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitWithoutPONR,41,1683694429655.b90fd344aff14f0728a80c3d09539e54.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1683694433500"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1683694433500"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1683694433500"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683694433500"}]},"ts":"1683694433500"} 2023-05-10 04:53:53,500 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitWithoutPONR 75e2592cb2ac08bd9f617bb9d0a0c896 2023-05-10 04:53:53,500 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitWithoutPONR,,1683694429655.75e2592cb2ac08bd9f617bb9d0a0c896.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-10 04:53:53,500 DEBUG [RS:2;jenkins-hbase4:37823-longCompactions-0] compactions.Compactor(207): Compacting 8edaf1b5392e42598afb9d4e334cf824.6c70954228a0bbe97dfdc76861367c66, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1683694428572 2023-05-10 04:53:53,500 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 75e2592cb2ac08bd9f617bb9d0a0c896 2023-05-10 04:53:53,500 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 75e2592cb2ac08bd9f617bb9d0a0c896 2023-05-10 04:53:53,501 DEBUG [RS:2;jenkins-hbase4:37823-longCompactions-0] compactions.Compactor(207): Compacting d5d0539ee2494349beff0d9e436cca78.6c70954228a0bbe97dfdc76861367c66, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1683694428782 2023-05-10 04:53:53,501 DEBUG [RS:2;jenkins-hbase4:37823-shortCompactions-0] compactions.Compactor(207): Compacting 94f3fbf75ed04880bf66a2015ae1796f.6c70954228a0bbe97dfdc76861367c66, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1683694428364 2023-05-10 04:53:53,502 DEBUG [RS:2;jenkins-hbase4:37823-shortCompactions-0] compactions.Compactor(207): Compacting 1db12d8725064a0e8efea5e9b8d4ee97.6c70954228a0bbe97dfdc76861367c66, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1683694428572 2023-05-10 04:53:53,502 DEBUG [RS:2;jenkins-hbase4:37823-longCompactions-0] compactions.Compactor(207): Compacting eff43a97e0c74de982adbbc3fbf1de59.6c70954228a0bbe97dfdc76861367c66, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1683694428985 2023-05-10 04:53:53,502 INFO [StoreOpener-75e2592cb2ac08bd9f617bb9d0a0c896-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 75e2592cb2ac08bd9f617bb9d0a0c896 2023-05-10 04:53:53,502 DEBUG [RS:2;jenkins-hbase4:37823-shortCompactions-0] compactions.Compactor(207): Compacting 6caf2d6b1504485d93a71a5fd6c7407b.6c70954228a0bbe97dfdc76861367c66, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1683694428782 2023-05-10 04:53:53,503 DEBUG [RS:2;jenkins-hbase4:37823-shortCompactions-0] compactions.Compactor(207): Compacting 79fdb2e1e5fb412d8fa298aca56435df.6c70954228a0bbe97dfdc76861367c66, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1683694428985 2023-05-10 04:53:53,503 DEBUG [RS:2;jenkins-hbase4:37823-longCompactions-0] compactions.Compactor(207): Compacting 7dc3c674095a4635b6b116c2f1bd84f1.6c70954228a0bbe97dfdc76861367c66, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1683694429188 2023-05-10 04:53:53,504 DEBUG [StoreOpener-75e2592cb2ac08bd9f617bb9d0a0c896-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/cf1 2023-05-10 04:53:53,504 DEBUG [RS:2;jenkins-hbase4:37823-shortCompactions-0] compactions.Compactor(207): Compacting 731aba47bdd0433abab5bc5a3bc06822.6c70954228a0bbe97dfdc76861367c66, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1683694429188 2023-05-10 04:53:53,504 DEBUG [StoreOpener-75e2592cb2ac08bd9f617bb9d0a0c896-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/cf1 2023-05-10 04:53:53,504 DEBUG [RS:2;jenkins-hbase4:37823-longCompactions-0] compactions.Compactor(207): Compacting 5dd4de4129d7484893b37527662845b3.6c70954228a0bbe97dfdc76861367c66, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1683694429392 2023-05-10 04:53:53,504 DEBUG [RS:2;jenkins-hbase4:37823-shortCompactions-0] compactions.Compactor(207): Compacting 6708ed14179848199d35a214bbb1223b.6c70954228a0bbe97dfdc76861367c66, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1683694429392 2023-05-10 04:53:53,504 INFO [StoreOpener-75e2592cb2ac08bd9f617bb9d0a0c896-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 75e2592cb2ac08bd9f617bb9d0a0c896 columnFamilyName cf1 2023-05-10 04:53:53,505 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=34, resume processing ppid=32 2023-05-10 04:53:53,505 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=34, ppid=32, state=SUCCESS; OpenRegionProcedure b90fd344aff14f0728a80c3d09539e54, server=jenkins-hbase4.apache.org,37823,1683694419534 in 261 msec 2023-05-10 04:53:53,507 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=32, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=b90fd344aff14f0728a80c3d09539e54, ASSIGN in 426 msec 2023-05-10 04:53:53,527 INFO [RS:2;jenkins-hbase4:37823-shortCompactions-0] throttle.PressureAwareThroughputController(145): b90fd344aff14f0728a80c3d09539e54#cf1#compaction#33 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-05-10 04:53:53,527 INFO [RS:2;jenkins-hbase4:37823-longCompactions-0] throttle.PressureAwareThroughputController(145): b90fd344aff14f0728a80c3d09539e54#cf2#compaction#32 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-05-10 04:53:53,532 DEBUG [StoreOpener-75e2592cb2ac08bd9f617bb9d0a0c896-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/cf1/1db12d8725064a0e8efea5e9b8d4ee97.6c70954228a0bbe97dfdc76861367c66->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf1/1db12d8725064a0e8efea5e9b8d4ee97-bottom 2023-05-10 04:53:53,536 DEBUG [StoreOpener-75e2592cb2ac08bd9f617bb9d0a0c896-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/cf1/6708ed14179848199d35a214bbb1223b.6c70954228a0bbe97dfdc76861367c66->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf1/6708ed14179848199d35a214bbb1223b-bottom 2023-05-10 04:53:53,544 DEBUG [StoreOpener-75e2592cb2ac08bd9f617bb9d0a0c896-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/cf1/692164c792dc453096c76676d73fe8e8.6c70954228a0bbe97dfdc76861367c66->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf1/692164c792dc453096c76676d73fe8e8-bottom 2023-05-10 04:53:53,558 DEBUG [StoreOpener-75e2592cb2ac08bd9f617bb9d0a0c896-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/cf1/6caf2d6b1504485d93a71a5fd6c7407b.6c70954228a0bbe97dfdc76861367c66->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf1/6caf2d6b1504485d93a71a5fd6c7407b-bottom 2023-05-10 04:53:53,568 DEBUG [StoreOpener-75e2592cb2ac08bd9f617bb9d0a0c896-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/cf1/731aba47bdd0433abab5bc5a3bc06822.6c70954228a0bbe97dfdc76861367c66->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf1/731aba47bdd0433abab5bc5a3bc06822-bottom 2023-05-10 04:53:53,576 DEBUG [StoreOpener-75e2592cb2ac08bd9f617bb9d0a0c896-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/cf1/79fdb2e1e5fb412d8fa298aca56435df.6c70954228a0bbe97dfdc76861367c66->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf1/79fdb2e1e5fb412d8fa298aca56435df-bottom 2023-05-10 04:53:53,581 DEBUG [RS:2;jenkins-hbase4:37823-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/.tmp/cf2/16c53043c6a34a7f8c542e569371cd2e as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/cf2/16c53043c6a34a7f8c542e569371cd2e 2023-05-10 04:53:53,584 DEBUG [RS:2;jenkins-hbase4:37823-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/.tmp/cf1/bf8c81c134c74c3f8df26efadf88cd16 as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/cf1/bf8c81c134c74c3f8df26efadf88cd16 2023-05-10 04:53:53,585 DEBUG [StoreOpener-75e2592cb2ac08bd9f617bb9d0a0c896-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/cf1/94f3fbf75ed04880bf66a2015ae1796f.6c70954228a0bbe97dfdc76861367c66->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf1/94f3fbf75ed04880bf66a2015ae1796f-bottom 2023-05-10 04:53:53,586 INFO [StoreOpener-75e2592cb2ac08bd9f617bb9d0a0c896-1] regionserver.HStore(310): Store=75e2592cb2ac08bd9f617bb9d0a0c896/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-10 04:53:53,586 INFO [StoreOpener-75e2592cb2ac08bd9f617bb9d0a0c896-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 75e2592cb2ac08bd9f617bb9d0a0c896 2023-05-10 04:53:53,587 DEBUG [StoreOpener-75e2592cb2ac08bd9f617bb9d0a0c896-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/cf2 2023-05-10 04:53:53,587 DEBUG [StoreOpener-75e2592cb2ac08bd9f617bb9d0a0c896-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/cf2 2023-05-10 04:53:53,588 INFO [StoreOpener-75e2592cb2ac08bd9f617bb9d0a0c896-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 75e2592cb2ac08bd9f617bb9d0a0c896 columnFamilyName cf2 2023-05-10 04:53:53,592 INFO [RS:2;jenkins-hbase4:37823-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in b90fd344aff14f0728a80c3d09539e54/cf1 of b90fd344aff14f0728a80c3d09539e54 into bf8c81c134c74c3f8df26efadf88cd16(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-05-10 04:53:53,593 DEBUG [RS:2;jenkins-hbase4:37823-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for b90fd344aff14f0728a80c3d09539e54: 2023-05-10 04:53:53,593 INFO [RS:2;jenkins-hbase4:37823-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,41,1683694429655.b90fd344aff14f0728a80c3d09539e54., storeName=b90fd344aff14f0728a80c3d09539e54/cf1, priority=9, startTime=1683694433495; duration=0sec 2023-05-10 04:53:53,593 DEBUG [RS:2;jenkins-hbase4:37823-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-10 04:53:53,597 INFO [RS:2;jenkins-hbase4:37823-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in b90fd344aff14f0728a80c3d09539e54/cf2 of b90fd344aff14f0728a80c3d09539e54 into 16c53043c6a34a7f8c542e569371cd2e(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-05-10 04:53:53,597 DEBUG [RS:2;jenkins-hbase4:37823-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for b90fd344aff14f0728a80c3d09539e54: 2023-05-10 04:53:53,597 INFO [RS:2;jenkins-hbase4:37823-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,41,1683694429655.b90fd344aff14f0728a80c3d09539e54., storeName=b90fd344aff14f0728a80c3d09539e54/cf2, priority=9, startTime=1683694433496; duration=0sec 2023-05-10 04:53:53,597 DEBUG [RS:2;jenkins-hbase4:37823-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-10 04:53:53,604 DEBUG [StoreOpener-75e2592cb2ac08bd9f617bb9d0a0c896-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/cf2/51d8a67670c645628aeb50ddbfc1a8dc.6c70954228a0bbe97dfdc76861367c66->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf2/51d8a67670c645628aeb50ddbfc1a8dc-bottom 2023-05-10 04:53:53,609 DEBUG [StoreOpener-75e2592cb2ac08bd9f617bb9d0a0c896-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/cf2/5b75cfdc8cfb4ced978b992b5b43f1be.6c70954228a0bbe97dfdc76861367c66->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf2/5b75cfdc8cfb4ced978b992b5b43f1be-bottom 2023-05-10 04:53:53,614 DEBUG [StoreOpener-75e2592cb2ac08bd9f617bb9d0a0c896-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/cf2/5dd4de4129d7484893b37527662845b3.6c70954228a0bbe97dfdc76861367c66->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf2/5dd4de4129d7484893b37527662845b3-bottom 2023-05-10 04:53:53,619 DEBUG [StoreOpener-75e2592cb2ac08bd9f617bb9d0a0c896-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/cf2/7dc3c674095a4635b6b116c2f1bd84f1.6c70954228a0bbe97dfdc76861367c66->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf2/7dc3c674095a4635b6b116c2f1bd84f1-bottom 2023-05-10 04:53:53,624 DEBUG [StoreOpener-75e2592cb2ac08bd9f617bb9d0a0c896-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/cf2/8edaf1b5392e42598afb9d4e334cf824.6c70954228a0bbe97dfdc76861367c66->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf2/8edaf1b5392e42598afb9d4e334cf824-bottom 2023-05-10 04:53:53,628 DEBUG [StoreOpener-75e2592cb2ac08bd9f617bb9d0a0c896-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/cf2/d5d0539ee2494349beff0d9e436cca78.6c70954228a0bbe97dfdc76861367c66->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf2/d5d0539ee2494349beff0d9e436cca78-bottom 2023-05-10 04:53:53,633 DEBUG [StoreOpener-75e2592cb2ac08bd9f617bb9d0a0c896-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/cf2/eff43a97e0c74de982adbbc3fbf1de59.6c70954228a0bbe97dfdc76861367c66->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf2/eff43a97e0c74de982adbbc3fbf1de59-bottom 2023-05-10 04:53:53,633 INFO [StoreOpener-75e2592cb2ac08bd9f617bb9d0a0c896-1] regionserver.HStore(310): Store=75e2592cb2ac08bd9f617bb9d0a0c896/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-10 04:53:53,634 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896 2023-05-10 04:53:53,634 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896 2023-05-10 04:53:53,636 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitWithoutPONR descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-05-10 04:53:53,637 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 75e2592cb2ac08bd9f617bb9d0a0c896 2023-05-10 04:53:53,637 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 75e2592cb2ac08bd9f617bb9d0a0c896; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67885545, jitterRate=0.011573448777198792}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-05-10 04:53:53,638 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 75e2592cb2ac08bd9f617bb9d0a0c896: 2023-05-10 04:53:53,638 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2334): Post open deploy tasks for testSplitWithoutPONR,,1683694429655.75e2592cb2ac08bd9f617bb9d0a0c896., pid=33, masterSystemTime=1683694433389 2023-05-10 04:53:53,638 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-10 04:53:53,640 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-05-10 04:53:53,640 DEBUG [RS:2;jenkins-hbase4:37823-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-05-10 04:53:53,640 DEBUG [RS:2;jenkins-hbase4:37823-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-05-10 04:53:53,640 INFO [RS:2;jenkins-hbase4:37823-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitWithoutPONR,,1683694429655.75e2592cb2ac08bd9f617bb9d0a0c896. 2023-05-10 04:53:53,640 INFO [RS:2;jenkins-hbase4:37823-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitWithoutPONR,,1683694429655.75e2592cb2ac08bd9f617bb9d0a0c896. 2023-05-10 04:53:53,640 DEBUG [RS:2;jenkins-hbase4:37823-shortCompactions-0] regionserver.HStore(1912): 75e2592cb2ac08bd9f617bb9d0a0c896/cf1 is initiating minor compaction (all files) 2023-05-10 04:53:53,640 DEBUG [RS:2;jenkins-hbase4:37823-longCompactions-0] regionserver.HStore(1912): 75e2592cb2ac08bd9f617bb9d0a0c896/cf2 is initiating minor compaction (all files) 2023-05-10 04:53:53,640 INFO [RS:2;jenkins-hbase4:37823-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 75e2592cb2ac08bd9f617bb9d0a0c896/cf1 in testSplitWithoutPONR,,1683694429655.75e2592cb2ac08bd9f617bb9d0a0c896. 2023-05-10 04:53:53,640 INFO [RS:2;jenkins-hbase4:37823-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 75e2592cb2ac08bd9f617bb9d0a0c896/cf2 in testSplitWithoutPONR,,1683694429655.75e2592cb2ac08bd9f617bb9d0a0c896. 2023-05-10 04:53:53,641 INFO [RS:2;jenkins-hbase4:37823-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/cf2/5b75cfdc8cfb4ced978b992b5b43f1be.6c70954228a0bbe97dfdc76861367c66->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf2/5b75cfdc8cfb4ced978b992b5b43f1be-bottom, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/cf2/51d8a67670c645628aeb50ddbfc1a8dc.6c70954228a0bbe97dfdc76861367c66->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf2/51d8a67670c645628aeb50ddbfc1a8dc-bottom, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/cf2/8edaf1b5392e42598afb9d4e334cf824.6c70954228a0bbe97dfdc76861367c66->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf2/8edaf1b5392e42598afb9d4e334cf824-bottom, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/cf2/d5d0539ee2494349beff0d9e436cca78.6c70954228a0bbe97dfdc76861367c66->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf2/d5d0539ee2494349beff0d9e436cca78-bottom, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/cf2/eff43a97e0c74de982adbbc3fbf1de59.6c70954228a0bbe97dfdc76861367c66->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf2/eff43a97e0c74de982adbbc3fbf1de59-bottom, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/cf2/7dc3c674095a4635b6b116c2f1bd84f1.6c70954228a0bbe97dfdc76861367c66->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf2/7dc3c674095a4635b6b116c2f1bd84f1-bottom, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/cf2/5dd4de4129d7484893b37527662845b3.6c70954228a0bbe97dfdc76861367c66->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf2/5dd4de4129d7484893b37527662845b3-bottom] into tmpdir=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/.tmp, totalSize=34.9 K 2023-05-10 04:53:53,641 INFO [RS:2;jenkins-hbase4:37823-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/cf1/692164c792dc453096c76676d73fe8e8.6c70954228a0bbe97dfdc76861367c66->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf1/692164c792dc453096c76676d73fe8e8-bottom, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/cf1/94f3fbf75ed04880bf66a2015ae1796f.6c70954228a0bbe97dfdc76861367c66->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf1/94f3fbf75ed04880bf66a2015ae1796f-bottom, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/cf1/1db12d8725064a0e8efea5e9b8d4ee97.6c70954228a0bbe97dfdc76861367c66->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf1/1db12d8725064a0e8efea5e9b8d4ee97-bottom, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/cf1/6caf2d6b1504485d93a71a5fd6c7407b.6c70954228a0bbe97dfdc76861367c66->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf1/6caf2d6b1504485d93a71a5fd6c7407b-bottom, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/cf1/79fdb2e1e5fb412d8fa298aca56435df.6c70954228a0bbe97dfdc76861367c66->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf1/79fdb2e1e5fb412d8fa298aca56435df-bottom, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/cf1/731aba47bdd0433abab5bc5a3bc06822.6c70954228a0bbe97dfdc76861367c66->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf1/731aba47bdd0433abab5bc5a3bc06822-bottom, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/cf1/6708ed14179848199d35a214bbb1223b.6c70954228a0bbe97dfdc76861367c66->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf1/6708ed14179848199d35a214bbb1223b-bottom] into tmpdir=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/.tmp, totalSize=34.9 K 2023-05-10 04:53:53,641 DEBUG [RS:2;jenkins-hbase4:37823-longCompactions-0] compactions.Compactor(207): Compacting 5b75cfdc8cfb4ced978b992b5b43f1be.6c70954228a0bbe97dfdc76861367c66, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1683694428155 2023-05-10 04:53:53,642 DEBUG [RS:2;jenkins-hbase4:37823-longCompactions-0] compactions.Compactor(207): Compacting 51d8a67670c645628aeb50ddbfc1a8dc.6c70954228a0bbe97dfdc76861367c66, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1683694428364 2023-05-10 04:53:53,642 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2361): Finished post open deploy task for testSplitWithoutPONR,,1683694429655.75e2592cb2ac08bd9f617bb9d0a0c896. 2023-05-10 04:53:53,642 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitWithoutPONR,,1683694429655.75e2592cb2ac08bd9f617bb9d0a0c896. 2023-05-10 04:53:53,642 DEBUG [RS:2;jenkins-hbase4:37823-shortCompactions-0] compactions.Compactor(207): Compacting 692164c792dc453096c76676d73fe8e8.6c70954228a0bbe97dfdc76861367c66, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1683694428155 2023-05-10 04:53:53,643 DEBUG [RS:2;jenkins-hbase4:37823-longCompactions-0] compactions.Compactor(207): Compacting 8edaf1b5392e42598afb9d4e334cf824.6c70954228a0bbe97dfdc76861367c66, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1683694428572 2023-05-10 04:53:53,643 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=31 updating hbase:meta row=75e2592cb2ac08bd9f617bb9d0a0c896, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:53,643 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitWithoutPONR,,1683694429655.75e2592cb2ac08bd9f617bb9d0a0c896.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1683694433643"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1683694433643"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1683694433643"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683694433643"}]},"ts":"1683694433643"} 2023-05-10 04:53:53,643 DEBUG [RS:2;jenkins-hbase4:37823-shortCompactions-0] compactions.Compactor(207): Compacting 94f3fbf75ed04880bf66a2015ae1796f.6c70954228a0bbe97dfdc76861367c66, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1683694428364 2023-05-10 04:53:53,643 DEBUG [RS:2;jenkins-hbase4:37823-longCompactions-0] compactions.Compactor(207): Compacting d5d0539ee2494349beff0d9e436cca78.6c70954228a0bbe97dfdc76861367c66, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1683694428782 2023-05-10 04:53:53,644 DEBUG [RS:2;jenkins-hbase4:37823-shortCompactions-0] compactions.Compactor(207): Compacting 1db12d8725064a0e8efea5e9b8d4ee97.6c70954228a0bbe97dfdc76861367c66, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1683694428572 2023-05-10 04:53:53,644 DEBUG [RS:2;jenkins-hbase4:37823-longCompactions-0] compactions.Compactor(207): Compacting eff43a97e0c74de982adbbc3fbf1de59.6c70954228a0bbe97dfdc76861367c66, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1683694428985 2023-05-10 04:53:53,644 DEBUG [RS:2;jenkins-hbase4:37823-shortCompactions-0] compactions.Compactor(207): Compacting 6caf2d6b1504485d93a71a5fd6c7407b.6c70954228a0bbe97dfdc76861367c66, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1683694428782 2023-05-10 04:53:53,645 DEBUG [RS:2;jenkins-hbase4:37823-shortCompactions-0] compactions.Compactor(207): Compacting 79fdb2e1e5fb412d8fa298aca56435df.6c70954228a0bbe97dfdc76861367c66, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1683694428985 2023-05-10 04:53:53,645 DEBUG [RS:2;jenkins-hbase4:37823-longCompactions-0] compactions.Compactor(207): Compacting 7dc3c674095a4635b6b116c2f1bd84f1.6c70954228a0bbe97dfdc76861367c66, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1683694429188 2023-05-10 04:53:53,646 DEBUG [RS:2;jenkins-hbase4:37823-shortCompactions-0] compactions.Compactor(207): Compacting 731aba47bdd0433abab5bc5a3bc06822.6c70954228a0bbe97dfdc76861367c66, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1683694429188 2023-05-10 04:53:53,646 DEBUG [RS:2;jenkins-hbase4:37823-longCompactions-0] compactions.Compactor(207): Compacting 5dd4de4129d7484893b37527662845b3.6c70954228a0bbe97dfdc76861367c66, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1683694429392 2023-05-10 04:53:53,647 DEBUG [RS:2;jenkins-hbase4:37823-shortCompactions-0] compactions.Compactor(207): Compacting 6708ed14179848199d35a214bbb1223b.6c70954228a0bbe97dfdc76861367c66, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1683694429392 2023-05-10 04:53:53,648 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=33, resume processing ppid=31 2023-05-10 04:53:53,648 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=33, ppid=31, state=SUCCESS; OpenRegionProcedure 75e2592cb2ac08bd9f617bb9d0a0c896, server=jenkins-hbase4.apache.org,37823,1683694419534 in 409 msec 2023-05-10 04:53:53,651 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=31, resume processing ppid=28 2023-05-10 04:53:53,651 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=31, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=75e2592cb2ac08bd9f617bb9d0a0c896, ASSIGN in 570 msec 2023-05-10 04:53:53,653 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6c70954228a0bbe97dfdc76861367c66, daughterA=75e2592cb2ac08bd9f617bb9d0a0c896, daughterB=b90fd344aff14f0728a80c3d09539e54 in 3.9960 sec 2023-05-10 04:53:53,664 INFO [RS:2;jenkins-hbase4:37823-shortCompactions-0] throttle.PressureAwareThroughputController(145): 75e2592cb2ac08bd9f617bb9d0a0c896#cf1#compaction#34 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-05-10 04:53:53,665 INFO [RS:2;jenkins-hbase4:37823-longCompactions-0] throttle.PressureAwareThroughputController(145): 75e2592cb2ac08bd9f617bb9d0a0c896#cf2#compaction#35 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-05-10 04:53:53,688 DEBUG [RS:2;jenkins-hbase4:37823-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/.tmp/cf1/05b565a814b2414495a5c5ec4518a15a as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/cf1/05b565a814b2414495a5c5ec4518a15a 2023-05-10 04:53:53,692 DEBUG [RS:2;jenkins-hbase4:37823-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/.tmp/cf2/0dcfd72e93b44ca0bfbfa0b0cb16bde6 as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/cf2/0dcfd72e93b44ca0bfbfa0b0cb16bde6 2023-05-10 04:53:53,694 INFO [RS:2;jenkins-hbase4:37823-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 75e2592cb2ac08bd9f617bb9d0a0c896/cf1 of 75e2592cb2ac08bd9f617bb9d0a0c896 into 05b565a814b2414495a5c5ec4518a15a(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-05-10 04:53:53,694 DEBUG [RS:2;jenkins-hbase4:37823-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 75e2592cb2ac08bd9f617bb9d0a0c896: 2023-05-10 04:53:53,694 INFO [RS:2;jenkins-hbase4:37823-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,,1683694429655.75e2592cb2ac08bd9f617bb9d0a0c896., storeName=75e2592cb2ac08bd9f617bb9d0a0c896/cf1, priority=9, startTime=1683694433638; duration=0sec 2023-05-10 04:53:53,694 DEBUG [RS:2;jenkins-hbase4:37823-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-10 04:53:53,699 INFO [RS:2;jenkins-hbase4:37823-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 75e2592cb2ac08bd9f617bb9d0a0c896/cf2 of 75e2592cb2ac08bd9f617bb9d0a0c896 into 0dcfd72e93b44ca0bfbfa0b0cb16bde6(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-05-10 04:53:53,699 DEBUG [RS:2;jenkins-hbase4:37823-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 75e2592cb2ac08bd9f617bb9d0a0c896: 2023-05-10 04:53:53,699 INFO [RS:2;jenkins-hbase4:37823-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,,1683694429655.75e2592cb2ac08bd9f617bb9d0a0c896., storeName=75e2592cb2ac08bd9f617bb9d0a0c896/cf2, priority=9, startTime=1683694433640; duration=0sec 2023-05-10 04:53:53,699 DEBUG [RS:2;jenkins-hbase4:37823-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-10 04:53:53,822 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37823] ipc.CallRunner(144): callId: 241 service: ClientService methodName: Scan size: 143 connection: 172.31.14.131:55416 deadline: 1683694493822, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitWithoutPONR,,1683694427650.6c70954228a0bbe97dfdc76861367c66. is not online on jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:53,936 DEBUG [Listener at localhost/45007] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-05-10 04:53:53,945 INFO [Listener at localhost/45007] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-05-10 04:53:53,947 DEBUG [Listener at localhost/45007] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-05-10 04:53:53,969 WARN [Listener at localhost/45007] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-05-10 04:53:53,971 INFO [Listener at localhost/45007] client.HBaseAdmin$15(890): Started disable of testSplitWithoutPONR 2023-05-10 04:53:53,972 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testSplitWithoutPONR 2023-05-10 04:53:53,973 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] procedure2.ProcedureExecutor(1029): Stored pid=35, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitWithoutPONR 2023-05-10 04:53:53,976 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-05-10 04:53:53,977 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683694433977"}]},"ts":"1683694433977"} 2023-05-10 04:53:53,979 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=DISABLING in hbase:meta 2023-05-10 04:53:53,980 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitWithoutPONR to state=DISABLING 2023-05-10 04:53:53,981 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=75e2592cb2ac08bd9f617bb9d0a0c896, UNASSIGN}, {pid=37, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=b90fd344aff14f0728a80c3d09539e54, UNASSIGN}] 2023-05-10 04:53:53,982 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=37, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=b90fd344aff14f0728a80c3d09539e54, UNASSIGN 2023-05-10 04:53:53,982 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=37 updating hbase:meta row=b90fd344aff14f0728a80c3d09539e54, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:53,982 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,41,1683694429655.b90fd344aff14f0728a80c3d09539e54.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1683694433982"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683694433982"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683694433982"}]},"ts":"1683694433982"} 2023-05-10 04:53:53,984 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=38, ppid=37, state=RUNNABLE; CloseRegionProcedure b90fd344aff14f0728a80c3d09539e54, server=jenkins-hbase4.apache.org,37823,1683694419534}] 2023-05-10 04:53:53,986 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=75e2592cb2ac08bd9f617bb9d0a0c896, UNASSIGN 2023-05-10 04:53:53,987 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=75e2592cb2ac08bd9f617bb9d0a0c896, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:53,987 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1683694429655.75e2592cb2ac08bd9f617bb9d0a0c896.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1683694433987"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683694433987"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683694433987"}]},"ts":"1683694433987"} 2023-05-10 04:53:53,988 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=39, ppid=36, state=RUNNABLE; CloseRegionProcedure 75e2592cb2ac08bd9f617bb9d0a0c896, server=jenkins-hbase4.apache.org,37823,1683694419534}] 2023-05-10 04:53:54,077 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-05-10 04:53:54,136 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 75e2592cb2ac08bd9f617bb9d0a0c896 2023-05-10 04:53:54,136 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 75e2592cb2ac08bd9f617bb9d0a0c896, disabling compactions & flushes 2023-05-10 04:53:54,136 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitWithoutPONR,,1683694429655.75e2592cb2ac08bd9f617bb9d0a0c896. 2023-05-10 04:53:54,136 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitWithoutPONR,,1683694429655.75e2592cb2ac08bd9f617bb9d0a0c896. 2023-05-10 04:53:54,136 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitWithoutPONR,,1683694429655.75e2592cb2ac08bd9f617bb9d0a0c896. after waiting 0 ms 2023-05-10 04:53:54,136 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitWithoutPONR,,1683694429655.75e2592cb2ac08bd9f617bb9d0a0c896. 2023-05-10 04:53:54,152 DEBUG [StoreCloser-testSplitWithoutPONR,,1683694429655.75e2592cb2ac08bd9f617bb9d0a0c896.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/cf1/692164c792dc453096c76676d73fe8e8.6c70954228a0bbe97dfdc76861367c66->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf1/692164c792dc453096c76676d73fe8e8-bottom, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/cf1/94f3fbf75ed04880bf66a2015ae1796f.6c70954228a0bbe97dfdc76861367c66->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf1/94f3fbf75ed04880bf66a2015ae1796f-bottom, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/cf1/1db12d8725064a0e8efea5e9b8d4ee97.6c70954228a0bbe97dfdc76861367c66->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf1/1db12d8725064a0e8efea5e9b8d4ee97-bottom, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/cf1/6caf2d6b1504485d93a71a5fd6c7407b.6c70954228a0bbe97dfdc76861367c66->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf1/6caf2d6b1504485d93a71a5fd6c7407b-bottom, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/cf1/79fdb2e1e5fb412d8fa298aca56435df.6c70954228a0bbe97dfdc76861367c66->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf1/79fdb2e1e5fb412d8fa298aca56435df-bottom, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/cf1/731aba47bdd0433abab5bc5a3bc06822.6c70954228a0bbe97dfdc76861367c66->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf1/731aba47bdd0433abab5bc5a3bc06822-bottom, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/cf1/6708ed14179848199d35a214bbb1223b.6c70954228a0bbe97dfdc76861367c66->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf1/6708ed14179848199d35a214bbb1223b-bottom] to archive 2023-05-10 04:53:54,153 DEBUG [StoreCloser-testSplitWithoutPONR,,1683694429655.75e2592cb2ac08bd9f617bb9d0a0c896.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-05-10 04:53:54,156 DEBUG [StoreCloser-testSplitWithoutPONR,,1683694429655.75e2592cb2ac08bd9f617bb9d0a0c896.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/cf1/692164c792dc453096c76676d73fe8e8.6c70954228a0bbe97dfdc76861367c66 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/cf1/692164c792dc453096c76676d73fe8e8.6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:54,157 DEBUG [StoreCloser-testSplitWithoutPONR,,1683694429655.75e2592cb2ac08bd9f617bb9d0a0c896.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/cf1/94f3fbf75ed04880bf66a2015ae1796f.6c70954228a0bbe97dfdc76861367c66 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/cf1/94f3fbf75ed04880bf66a2015ae1796f.6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:54,159 DEBUG [StoreCloser-testSplitWithoutPONR,,1683694429655.75e2592cb2ac08bd9f617bb9d0a0c896.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/cf1/1db12d8725064a0e8efea5e9b8d4ee97.6c70954228a0bbe97dfdc76861367c66 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/cf1/1db12d8725064a0e8efea5e9b8d4ee97.6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:54,160 DEBUG [StoreCloser-testSplitWithoutPONR,,1683694429655.75e2592cb2ac08bd9f617bb9d0a0c896.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/cf1/6caf2d6b1504485d93a71a5fd6c7407b.6c70954228a0bbe97dfdc76861367c66 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/cf1/6caf2d6b1504485d93a71a5fd6c7407b.6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:54,161 DEBUG [StoreCloser-testSplitWithoutPONR,,1683694429655.75e2592cb2ac08bd9f617bb9d0a0c896.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/cf1/79fdb2e1e5fb412d8fa298aca56435df.6c70954228a0bbe97dfdc76861367c66 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/cf1/79fdb2e1e5fb412d8fa298aca56435df.6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:54,162 DEBUG [StoreCloser-testSplitWithoutPONR,,1683694429655.75e2592cb2ac08bd9f617bb9d0a0c896.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/cf1/731aba47bdd0433abab5bc5a3bc06822.6c70954228a0bbe97dfdc76861367c66 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/cf1/731aba47bdd0433abab5bc5a3bc06822.6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:54,164 DEBUG [StoreCloser-testSplitWithoutPONR,,1683694429655.75e2592cb2ac08bd9f617bb9d0a0c896.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/cf1/6708ed14179848199d35a214bbb1223b.6c70954228a0bbe97dfdc76861367c66 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/cf1/6708ed14179848199d35a214bbb1223b.6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:54,166 DEBUG [StoreCloser-testSplitWithoutPONR,,1683694429655.75e2592cb2ac08bd9f617bb9d0a0c896.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/cf2/5b75cfdc8cfb4ced978b992b5b43f1be.6c70954228a0bbe97dfdc76861367c66->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf2/5b75cfdc8cfb4ced978b992b5b43f1be-bottom, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/cf2/51d8a67670c645628aeb50ddbfc1a8dc.6c70954228a0bbe97dfdc76861367c66->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf2/51d8a67670c645628aeb50ddbfc1a8dc-bottom, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/cf2/8edaf1b5392e42598afb9d4e334cf824.6c70954228a0bbe97dfdc76861367c66->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf2/8edaf1b5392e42598afb9d4e334cf824-bottom, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/cf2/d5d0539ee2494349beff0d9e436cca78.6c70954228a0bbe97dfdc76861367c66->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf2/d5d0539ee2494349beff0d9e436cca78-bottom, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/cf2/eff43a97e0c74de982adbbc3fbf1de59.6c70954228a0bbe97dfdc76861367c66->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf2/eff43a97e0c74de982adbbc3fbf1de59-bottom, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/cf2/7dc3c674095a4635b6b116c2f1bd84f1.6c70954228a0bbe97dfdc76861367c66->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf2/7dc3c674095a4635b6b116c2f1bd84f1-bottom, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/cf2/5dd4de4129d7484893b37527662845b3.6c70954228a0bbe97dfdc76861367c66->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf2/5dd4de4129d7484893b37527662845b3-bottom] to archive 2023-05-10 04:53:54,167 DEBUG [StoreCloser-testSplitWithoutPONR,,1683694429655.75e2592cb2ac08bd9f617bb9d0a0c896.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-05-10 04:53:54,169 DEBUG [StoreCloser-testSplitWithoutPONR,,1683694429655.75e2592cb2ac08bd9f617bb9d0a0c896.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/cf2/5b75cfdc8cfb4ced978b992b5b43f1be.6c70954228a0bbe97dfdc76861367c66 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/cf2/5b75cfdc8cfb4ced978b992b5b43f1be.6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:54,171 DEBUG [StoreCloser-testSplitWithoutPONR,,1683694429655.75e2592cb2ac08bd9f617bb9d0a0c896.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/cf2/51d8a67670c645628aeb50ddbfc1a8dc.6c70954228a0bbe97dfdc76861367c66 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/cf2/51d8a67670c645628aeb50ddbfc1a8dc.6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:54,172 DEBUG [StoreCloser-testSplitWithoutPONR,,1683694429655.75e2592cb2ac08bd9f617bb9d0a0c896.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/cf2/8edaf1b5392e42598afb9d4e334cf824.6c70954228a0bbe97dfdc76861367c66 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/cf2/8edaf1b5392e42598afb9d4e334cf824.6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:54,173 DEBUG [StoreCloser-testSplitWithoutPONR,,1683694429655.75e2592cb2ac08bd9f617bb9d0a0c896.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/cf2/d5d0539ee2494349beff0d9e436cca78.6c70954228a0bbe97dfdc76861367c66 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/cf2/d5d0539ee2494349beff0d9e436cca78.6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:54,174 DEBUG [StoreCloser-testSplitWithoutPONR,,1683694429655.75e2592cb2ac08bd9f617bb9d0a0c896.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/cf2/eff43a97e0c74de982adbbc3fbf1de59.6c70954228a0bbe97dfdc76861367c66 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/cf2/eff43a97e0c74de982adbbc3fbf1de59.6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:54,175 DEBUG [StoreCloser-testSplitWithoutPONR,,1683694429655.75e2592cb2ac08bd9f617bb9d0a0c896.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/cf2/7dc3c674095a4635b6b116c2f1bd84f1.6c70954228a0bbe97dfdc76861367c66 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/cf2/7dc3c674095a4635b6b116c2f1bd84f1.6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:54,176 DEBUG [StoreCloser-testSplitWithoutPONR,,1683694429655.75e2592cb2ac08bd9f617bb9d0a0c896.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/cf2/5dd4de4129d7484893b37527662845b3.6c70954228a0bbe97dfdc76861367c66 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/cf2/5dd4de4129d7484893b37527662845b3.6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:54,180 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-05-10 04:53:54,181 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-10 04:53:54,181 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitWithoutPONR,,1683694429655.75e2592cb2ac08bd9f617bb9d0a0c896. 2023-05-10 04:53:54,181 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 75e2592cb2ac08bd9f617bb9d0a0c896: 2023-05-10 04:53:54,183 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 75e2592cb2ac08bd9f617bb9d0a0c896 2023-05-10 04:53:54,183 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close b90fd344aff14f0728a80c3d09539e54 2023-05-10 04:53:54,183 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing b90fd344aff14f0728a80c3d09539e54, disabling compactions & flushes 2023-05-10 04:53:54,183 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitWithoutPONR,41,1683694429655.b90fd344aff14f0728a80c3d09539e54. 2023-05-10 04:53:54,183 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitWithoutPONR,41,1683694429655.b90fd344aff14f0728a80c3d09539e54. 2023-05-10 04:53:54,183 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitWithoutPONR,41,1683694429655.b90fd344aff14f0728a80c3d09539e54. after waiting 0 ms 2023-05-10 04:53:54,183 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitWithoutPONR,41,1683694429655.b90fd344aff14f0728a80c3d09539e54. 2023-05-10 04:53:54,183 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=75e2592cb2ac08bd9f617bb9d0a0c896, regionState=CLOSED 2023-05-10 04:53:54,184 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitWithoutPONR,,1683694429655.75e2592cb2ac08bd9f617bb9d0a0c896.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1683694434183"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683694434183"}]},"ts":"1683694434183"} 2023-05-10 04:53:54,184 DEBUG [StoreCloser-testSplitWithoutPONR,41,1683694429655.b90fd344aff14f0728a80c3d09539e54.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/cf1/692164c792dc453096c76676d73fe8e8.6c70954228a0bbe97dfdc76861367c66->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf1/692164c792dc453096c76676d73fe8e8-top, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/cf1/94f3fbf75ed04880bf66a2015ae1796f.6c70954228a0bbe97dfdc76861367c66->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf1/94f3fbf75ed04880bf66a2015ae1796f-top, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/cf1/1db12d8725064a0e8efea5e9b8d4ee97.6c70954228a0bbe97dfdc76861367c66->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf1/1db12d8725064a0e8efea5e9b8d4ee97-top, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/cf1/6caf2d6b1504485d93a71a5fd6c7407b.6c70954228a0bbe97dfdc76861367c66->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf1/6caf2d6b1504485d93a71a5fd6c7407b-top, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/cf1/79fdb2e1e5fb412d8fa298aca56435df.6c70954228a0bbe97dfdc76861367c66->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf1/79fdb2e1e5fb412d8fa298aca56435df-top, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/cf1/731aba47bdd0433abab5bc5a3bc06822.6c70954228a0bbe97dfdc76861367c66->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf1/731aba47bdd0433abab5bc5a3bc06822-top, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/cf1/6708ed14179848199d35a214bbb1223b.6c70954228a0bbe97dfdc76861367c66->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf1/6708ed14179848199d35a214bbb1223b-top] to archive 2023-05-10 04:53:54,185 DEBUG [StoreCloser-testSplitWithoutPONR,41,1683694429655.b90fd344aff14f0728a80c3d09539e54.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-05-10 04:53:54,188 DEBUG [StoreCloser-testSplitWithoutPONR,41,1683694429655.b90fd344aff14f0728a80c3d09539e54.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/cf1/692164c792dc453096c76676d73fe8e8.6c70954228a0bbe97dfdc76861367c66 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/cf1/692164c792dc453096c76676d73fe8e8.6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:54,188 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=39, resume processing ppid=36 2023-05-10 04:53:54,189 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=39, ppid=36, state=SUCCESS; CloseRegionProcedure 75e2592cb2ac08bd9f617bb9d0a0c896, server=jenkins-hbase4.apache.org,37823,1683694419534 in 198 msec 2023-05-10 04:53:54,190 DEBUG [StoreCloser-testSplitWithoutPONR,41,1683694429655.b90fd344aff14f0728a80c3d09539e54.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/cf1/94f3fbf75ed04880bf66a2015ae1796f.6c70954228a0bbe97dfdc76861367c66 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/cf1/94f3fbf75ed04880bf66a2015ae1796f.6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:54,191 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=36, ppid=35, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=75e2592cb2ac08bd9f617bb9d0a0c896, UNASSIGN in 208 msec 2023-05-10 04:53:54,192 DEBUG [StoreCloser-testSplitWithoutPONR,41,1683694429655.b90fd344aff14f0728a80c3d09539e54.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/cf1/1db12d8725064a0e8efea5e9b8d4ee97.6c70954228a0bbe97dfdc76861367c66 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/cf1/1db12d8725064a0e8efea5e9b8d4ee97.6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:54,193 DEBUG [StoreCloser-testSplitWithoutPONR,41,1683694429655.b90fd344aff14f0728a80c3d09539e54.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/cf1/6caf2d6b1504485d93a71a5fd6c7407b.6c70954228a0bbe97dfdc76861367c66 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/cf1/6caf2d6b1504485d93a71a5fd6c7407b.6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:54,195 DEBUG [StoreCloser-testSplitWithoutPONR,41,1683694429655.b90fd344aff14f0728a80c3d09539e54.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/cf1/79fdb2e1e5fb412d8fa298aca56435df.6c70954228a0bbe97dfdc76861367c66 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/cf1/79fdb2e1e5fb412d8fa298aca56435df.6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:54,196 DEBUG [StoreCloser-testSplitWithoutPONR,41,1683694429655.b90fd344aff14f0728a80c3d09539e54.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/cf1/731aba47bdd0433abab5bc5a3bc06822.6c70954228a0bbe97dfdc76861367c66 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/cf1/731aba47bdd0433abab5bc5a3bc06822.6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:54,198 DEBUG [StoreCloser-testSplitWithoutPONR,41,1683694429655.b90fd344aff14f0728a80c3d09539e54.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/cf1/6708ed14179848199d35a214bbb1223b.6c70954228a0bbe97dfdc76861367c66 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/cf1/6708ed14179848199d35a214bbb1223b.6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:54,200 DEBUG [StoreCloser-testSplitWithoutPONR,41,1683694429655.b90fd344aff14f0728a80c3d09539e54.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/cf2/5b75cfdc8cfb4ced978b992b5b43f1be.6c70954228a0bbe97dfdc76861367c66->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf2/5b75cfdc8cfb4ced978b992b5b43f1be-top, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/cf2/51d8a67670c645628aeb50ddbfc1a8dc.6c70954228a0bbe97dfdc76861367c66->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf2/51d8a67670c645628aeb50ddbfc1a8dc-top, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/cf2/8edaf1b5392e42598afb9d4e334cf824.6c70954228a0bbe97dfdc76861367c66->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf2/8edaf1b5392e42598afb9d4e334cf824-top, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/cf2/d5d0539ee2494349beff0d9e436cca78.6c70954228a0bbe97dfdc76861367c66->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf2/d5d0539ee2494349beff0d9e436cca78-top, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/cf2/eff43a97e0c74de982adbbc3fbf1de59.6c70954228a0bbe97dfdc76861367c66->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf2/eff43a97e0c74de982adbbc3fbf1de59-top, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/cf2/7dc3c674095a4635b6b116c2f1bd84f1.6c70954228a0bbe97dfdc76861367c66->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf2/7dc3c674095a4635b6b116c2f1bd84f1-top, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/cf2/5dd4de4129d7484893b37527662845b3.6c70954228a0bbe97dfdc76861367c66->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf2/5dd4de4129d7484893b37527662845b3-top] to archive 2023-05-10 04:53:54,201 DEBUG [StoreCloser-testSplitWithoutPONR,41,1683694429655.b90fd344aff14f0728a80c3d09539e54.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-05-10 04:53:54,202 DEBUG [StoreCloser-testSplitWithoutPONR,41,1683694429655.b90fd344aff14f0728a80c3d09539e54.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/cf2/5b75cfdc8cfb4ced978b992b5b43f1be.6c70954228a0bbe97dfdc76861367c66 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/cf2/5b75cfdc8cfb4ced978b992b5b43f1be.6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:54,204 DEBUG [StoreCloser-testSplitWithoutPONR,41,1683694429655.b90fd344aff14f0728a80c3d09539e54.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/cf2/51d8a67670c645628aeb50ddbfc1a8dc.6c70954228a0bbe97dfdc76861367c66 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/cf2/51d8a67670c645628aeb50ddbfc1a8dc.6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:54,205 DEBUG [StoreCloser-testSplitWithoutPONR,41,1683694429655.b90fd344aff14f0728a80c3d09539e54.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/cf2/8edaf1b5392e42598afb9d4e334cf824.6c70954228a0bbe97dfdc76861367c66 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/cf2/8edaf1b5392e42598afb9d4e334cf824.6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:54,206 DEBUG [StoreCloser-testSplitWithoutPONR,41,1683694429655.b90fd344aff14f0728a80c3d09539e54.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/cf2/d5d0539ee2494349beff0d9e436cca78.6c70954228a0bbe97dfdc76861367c66 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/cf2/d5d0539ee2494349beff0d9e436cca78.6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:54,207 DEBUG [StoreCloser-testSplitWithoutPONR,41,1683694429655.b90fd344aff14f0728a80c3d09539e54.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/cf2/eff43a97e0c74de982adbbc3fbf1de59.6c70954228a0bbe97dfdc76861367c66 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/cf2/eff43a97e0c74de982adbbc3fbf1de59.6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:54,209 DEBUG [StoreCloser-testSplitWithoutPONR,41,1683694429655.b90fd344aff14f0728a80c3d09539e54.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/cf2/7dc3c674095a4635b6b116c2f1bd84f1.6c70954228a0bbe97dfdc76861367c66 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/cf2/7dc3c674095a4635b6b116c2f1bd84f1.6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:54,210 DEBUG [StoreCloser-testSplitWithoutPONR,41,1683694429655.b90fd344aff14f0728a80c3d09539e54.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/cf2/5dd4de4129d7484893b37527662845b3.6c70954228a0bbe97dfdc76861367c66 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/cf2/5dd4de4129d7484893b37527662845b3.6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:54,214 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-05-10 04:53:54,215 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-10 04:53:54,215 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitWithoutPONR,41,1683694429655.b90fd344aff14f0728a80c3d09539e54. 2023-05-10 04:53:54,215 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for b90fd344aff14f0728a80c3d09539e54: 2023-05-10 04:53:54,217 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed b90fd344aff14f0728a80c3d09539e54 2023-05-10 04:53:54,217 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=37 updating hbase:meta row=b90fd344aff14f0728a80c3d09539e54, regionState=CLOSED 2023-05-10 04:53:54,217 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitWithoutPONR,41,1683694429655.b90fd344aff14f0728a80c3d09539e54.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1683694434217"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683694434217"}]},"ts":"1683694434217"} 2023-05-10 04:53:54,221 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=38, resume processing ppid=37 2023-05-10 04:53:54,221 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=38, ppid=37, state=SUCCESS; CloseRegionProcedure b90fd344aff14f0728a80c3d09539e54, server=jenkins-hbase4.apache.org,37823,1683694419534 in 235 msec 2023-05-10 04:53:54,223 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=37, resume processing ppid=35 2023-05-10 04:53:54,223 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=37, ppid=35, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=b90fd344aff14f0728a80c3d09539e54, UNASSIGN in 240 msec 2023-05-10 04:53:54,224 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683694434224"}]},"ts":"1683694434224"} 2023-05-10 04:53:54,226 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=DISABLED in hbase:meta 2023-05-10 04:53:54,228 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitWithoutPONR to state=DISABLED 2023-05-10 04:53:54,230 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR in 256 msec 2023-05-10 04:53:54,279 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(1227): Checking to see if procedure is done pid=35 2023-05-10 04:53:54,279 INFO [Listener at localhost/45007] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitWithoutPONR, procId: 35 completed 2023-05-10 04:53:54,280 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete testSplitWithoutPONR 2023-05-10 04:53:54,281 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] procedure2.ProcedureExecutor(1029): Stored pid=40, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-10 04:53:54,283 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=40, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-10 04:53:54,284 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=40, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-10 04:53:54,284 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(1227): Checking to see if procedure is done pid=40 2023-05-10 04:53:54,292 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:54,292 DEBUG [HFileArchiver-8] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896 2023-05-10 04:53:54,292 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54 2023-05-10 04:53:54,295 DEBUG [HFileArchiver-7] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf1, FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf2, FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/recovered.edits] 2023-05-10 04:53:54,295 DEBUG [HFileArchiver-8] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/75e2592cb2ac08bd9f617bb9d0a0c896, FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/cf1, FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/cf2, FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/recovered.edits] 2023-05-10 04:53:54,295 DEBUG [HFileArchiver-1] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/b90fd344aff14f0728a80c3d09539e54, FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/cf1, FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/cf2, FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/recovered.edits] 2023-05-10 04:53:54,299 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/75e2592cb2ac08bd9f617bb9d0a0c896/.regioninfo to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/75e2592cb2ac08bd9f617bb9d0a0c896/.regioninfo 2023-05-10 04:53:54,299 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/b90fd344aff14f0728a80c3d09539e54/.regioninfo to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/b90fd344aff14f0728a80c3d09539e54/.regioninfo 2023-05-10 04:53:54,299 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf1/1db12d8725064a0e8efea5e9b8d4ee97 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf1/1db12d8725064a0e8efea5e9b8d4ee97 2023-05-10 04:53:54,301 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf1/6708ed14179848199d35a214bbb1223b to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf1/6708ed14179848199d35a214bbb1223b 2023-05-10 04:53:54,303 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/75e2592cb2ac08bd9f617bb9d0a0c896/cf1/1db12d8725064a0e8efea5e9b8d4ee97.6c70954228a0bbe97dfdc76861367c66 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/75e2592cb2ac08bd9f617bb9d0a0c896/cf1/1db12d8725064a0e8efea5e9b8d4ee97.6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:54,303 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf1/692164c792dc453096c76676d73fe8e8 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf1/692164c792dc453096c76676d73fe8e8 2023-05-10 04:53:54,304 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/b90fd344aff14f0728a80c3d09539e54/cf1/1db12d8725064a0e8efea5e9b8d4ee97.6c70954228a0bbe97dfdc76861367c66 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/b90fd344aff14f0728a80c3d09539e54/cf1/1db12d8725064a0e8efea5e9b8d4ee97.6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:54,307 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf1/6caf2d6b1504485d93a71a5fd6c7407b to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf1/6caf2d6b1504485d93a71a5fd6c7407b 2023-05-10 04:53:54,307 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/75e2592cb2ac08bd9f617bb9d0a0c896/cf1/6708ed14179848199d35a214bbb1223b.6c70954228a0bbe97dfdc76861367c66 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/75e2592cb2ac08bd9f617bb9d0a0c896/cf1/6708ed14179848199d35a214bbb1223b.6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:54,308 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/b90fd344aff14f0728a80c3d09539e54/cf1/6708ed14179848199d35a214bbb1223b.6c70954228a0bbe97dfdc76861367c66 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/b90fd344aff14f0728a80c3d09539e54/cf1/6708ed14179848199d35a214bbb1223b.6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:54,309 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf1/731aba47bdd0433abab5bc5a3bc06822 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf1/731aba47bdd0433abab5bc5a3bc06822 2023-05-10 04:53:54,309 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/75e2592cb2ac08bd9f617bb9d0a0c896/cf1/692164c792dc453096c76676d73fe8e8.6c70954228a0bbe97dfdc76861367c66 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/75e2592cb2ac08bd9f617bb9d0a0c896/cf1/692164c792dc453096c76676d73fe8e8.6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:54,310 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/b90fd344aff14f0728a80c3d09539e54/cf1/692164c792dc453096c76676d73fe8e8.6c70954228a0bbe97dfdc76861367c66 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/b90fd344aff14f0728a80c3d09539e54/cf1/692164c792dc453096c76676d73fe8e8.6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:54,310 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf1/79fdb2e1e5fb412d8fa298aca56435df to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf1/79fdb2e1e5fb412d8fa298aca56435df 2023-05-10 04:53:54,311 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/75e2592cb2ac08bd9f617bb9d0a0c896/cf1/6caf2d6b1504485d93a71a5fd6c7407b.6c70954228a0bbe97dfdc76861367c66 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/75e2592cb2ac08bd9f617bb9d0a0c896/cf1/6caf2d6b1504485d93a71a5fd6c7407b.6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:54,311 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/b90fd344aff14f0728a80c3d09539e54/cf1/6caf2d6b1504485d93a71a5fd6c7407b.6c70954228a0bbe97dfdc76861367c66 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/b90fd344aff14f0728a80c3d09539e54/cf1/6caf2d6b1504485d93a71a5fd6c7407b.6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:54,312 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf1/94f3fbf75ed04880bf66a2015ae1796f to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf1/94f3fbf75ed04880bf66a2015ae1796f 2023-05-10 04:53:54,313 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/75e2592cb2ac08bd9f617bb9d0a0c896/cf1/731aba47bdd0433abab5bc5a3bc06822.6c70954228a0bbe97dfdc76861367c66 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/75e2592cb2ac08bd9f617bb9d0a0c896/cf1/731aba47bdd0433abab5bc5a3bc06822.6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:54,313 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/b90fd344aff14f0728a80c3d09539e54/cf1/731aba47bdd0433abab5bc5a3bc06822.6c70954228a0bbe97dfdc76861367c66 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/b90fd344aff14f0728a80c3d09539e54/cf1/731aba47bdd0433abab5bc5a3bc06822.6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:54,315 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/b90fd344aff14f0728a80c3d09539e54/cf1/79fdb2e1e5fb412d8fa298aca56435df.6c70954228a0bbe97dfdc76861367c66 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/b90fd344aff14f0728a80c3d09539e54/cf1/79fdb2e1e5fb412d8fa298aca56435df.6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:54,315 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/75e2592cb2ac08bd9f617bb9d0a0c896/cf1/79fdb2e1e5fb412d8fa298aca56435df.6c70954228a0bbe97dfdc76861367c66 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/75e2592cb2ac08bd9f617bb9d0a0c896/cf1/79fdb2e1e5fb412d8fa298aca56435df.6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:54,316 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/b90fd344aff14f0728a80c3d09539e54/cf1/94f3fbf75ed04880bf66a2015ae1796f.6c70954228a0bbe97dfdc76861367c66 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/b90fd344aff14f0728a80c3d09539e54/cf1/94f3fbf75ed04880bf66a2015ae1796f.6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:54,317 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf2/51d8a67670c645628aeb50ddbfc1a8dc to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf2/51d8a67670c645628aeb50ddbfc1a8dc 2023-05-10 04:53:54,317 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/75e2592cb2ac08bd9f617bb9d0a0c896/cf1/94f3fbf75ed04880bf66a2015ae1796f.6c70954228a0bbe97dfdc76861367c66 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/75e2592cb2ac08bd9f617bb9d0a0c896/cf1/94f3fbf75ed04880bf66a2015ae1796f.6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:54,319 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf2/5b75cfdc8cfb4ced978b992b5b43f1be to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf2/5b75cfdc8cfb4ced978b992b5b43f1be 2023-05-10 04:53:54,321 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf2/5dd4de4129d7484893b37527662845b3 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf2/5dd4de4129d7484893b37527662845b3 2023-05-10 04:53:54,321 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/b90fd344aff14f0728a80c3d09539e54/cf2/51d8a67670c645628aeb50ddbfc1a8dc.6c70954228a0bbe97dfdc76861367c66 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/b90fd344aff14f0728a80c3d09539e54/cf2/51d8a67670c645628aeb50ddbfc1a8dc.6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:54,322 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/75e2592cb2ac08bd9f617bb9d0a0c896/cf2/51d8a67670c645628aeb50ddbfc1a8dc.6c70954228a0bbe97dfdc76861367c66 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/75e2592cb2ac08bd9f617bb9d0a0c896/cf2/51d8a67670c645628aeb50ddbfc1a8dc.6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:54,324 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/b90fd344aff14f0728a80c3d09539e54/cf2/5b75cfdc8cfb4ced978b992b5b43f1be.6c70954228a0bbe97dfdc76861367c66 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/b90fd344aff14f0728a80c3d09539e54/cf2/5b75cfdc8cfb4ced978b992b5b43f1be.6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:54,324 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf2/7dc3c674095a4635b6b116c2f1bd84f1 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf2/7dc3c674095a4635b6b116c2f1bd84f1 2023-05-10 04:53:54,325 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/75e2592cb2ac08bd9f617bb9d0a0c896/cf2/5b75cfdc8cfb4ced978b992b5b43f1be.6c70954228a0bbe97dfdc76861367c66 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/75e2592cb2ac08bd9f617bb9d0a0c896/cf2/5b75cfdc8cfb4ced978b992b5b43f1be.6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:54,334 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/b90fd344aff14f0728a80c3d09539e54/cf2/5dd4de4129d7484893b37527662845b3.6c70954228a0bbe97dfdc76861367c66 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/b90fd344aff14f0728a80c3d09539e54/cf2/5dd4de4129d7484893b37527662845b3.6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:54,334 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf2/8edaf1b5392e42598afb9d4e334cf824 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf2/8edaf1b5392e42598afb9d4e334cf824 2023-05-10 04:53:54,334 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/75e2592cb2ac08bd9f617bb9d0a0c896/cf2/5dd4de4129d7484893b37527662845b3.6c70954228a0bbe97dfdc76861367c66 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/75e2592cb2ac08bd9f617bb9d0a0c896/cf2/5dd4de4129d7484893b37527662845b3.6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:54,337 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/b90fd344aff14f0728a80c3d09539e54/cf2/7dc3c674095a4635b6b116c2f1bd84f1.6c70954228a0bbe97dfdc76861367c66 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/b90fd344aff14f0728a80c3d09539e54/cf2/7dc3c674095a4635b6b116c2f1bd84f1.6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:54,337 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf2/d5d0539ee2494349beff0d9e436cca78 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf2/d5d0539ee2494349beff0d9e436cca78 2023-05-10 04:53:54,338 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/75e2592cb2ac08bd9f617bb9d0a0c896/cf2/7dc3c674095a4635b6b116c2f1bd84f1.6c70954228a0bbe97dfdc76861367c66 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/75e2592cb2ac08bd9f617bb9d0a0c896/cf2/7dc3c674095a4635b6b116c2f1bd84f1.6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:54,339 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf2/eff43a97e0c74de982adbbc3fbf1de59 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/cf2/eff43a97e0c74de982adbbc3fbf1de59 2023-05-10 04:53:54,339 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/b90fd344aff14f0728a80c3d09539e54/cf2/8edaf1b5392e42598afb9d4e334cf824.6c70954228a0bbe97dfdc76861367c66 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/b90fd344aff14f0728a80c3d09539e54/cf2/8edaf1b5392e42598afb9d4e334cf824.6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:54,339 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/75e2592cb2ac08bd9f617bb9d0a0c896/cf2/8edaf1b5392e42598afb9d4e334cf824.6c70954228a0bbe97dfdc76861367c66 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/75e2592cb2ac08bd9f617bb9d0a0c896/cf2/8edaf1b5392e42598afb9d4e334cf824.6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:54,341 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/b90fd344aff14f0728a80c3d09539e54/cf2/d5d0539ee2494349beff0d9e436cca78.6c70954228a0bbe97dfdc76861367c66 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/b90fd344aff14f0728a80c3d09539e54/cf2/d5d0539ee2494349beff0d9e436cca78.6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:54,341 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/75e2592cb2ac08bd9f617bb9d0a0c896/cf2/d5d0539ee2494349beff0d9e436cca78.6c70954228a0bbe97dfdc76861367c66 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/75e2592cb2ac08bd9f617bb9d0a0c896/cf2/d5d0539ee2494349beff0d9e436cca78.6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:54,342 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/b90fd344aff14f0728a80c3d09539e54/cf2/eff43a97e0c74de982adbbc3fbf1de59.6c70954228a0bbe97dfdc76861367c66 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/b90fd344aff14f0728a80c3d09539e54/cf2/eff43a97e0c74de982adbbc3fbf1de59.6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:54,342 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/recovered.edits/85.seqid to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66/recovered.edits/85.seqid 2023-05-10 04:53:54,343 DEBUG [HFileArchiver-7] backup.HFileArchiver(596): Deleted hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitWithoutPONR/6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:54,345 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/75e2592cb2ac08bd9f617bb9d0a0c896/cf2/eff43a97e0c74de982adbbc3fbf1de59.6c70954228a0bbe97dfdc76861367c66 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/75e2592cb2ac08bd9f617bb9d0a0c896/cf2/eff43a97e0c74de982adbbc3fbf1de59.6c70954228a0bbe97dfdc76861367c66 2023-05-10 04:53:54,345 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/cf1/bf8c81c134c74c3f8df26efadf88cd16 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/cf1/bf8c81c134c74c3f8df26efadf88cd16 2023-05-10 04:53:54,348 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/cf1/05b565a814b2414495a5c5ec4518a15a to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/cf1/05b565a814b2414495a5c5ec4518a15a 2023-05-10 04:53:54,348 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/cf2/16c53043c6a34a7f8c542e569371cd2e to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/cf2/16c53043c6a34a7f8c542e569371cd2e 2023-05-10 04:53:54,350 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/cf2/0dcfd72e93b44ca0bfbfa0b0cb16bde6 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/cf2/0dcfd72e93b44ca0bfbfa0b0cb16bde6 2023-05-10 04:53:54,351 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/recovered.edits/90.seqid to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54/recovered.edits/90.seqid 2023-05-10 04:53:54,352 DEBUG [HFileArchiver-1] backup.HFileArchiver(596): Deleted hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitWithoutPONR/b90fd344aff14f0728a80c3d09539e54 2023-05-10 04:53:54,353 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/recovered.edits/90.seqid to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896/recovered.edits/90.seqid 2023-05-10 04:53:54,354 DEBUG [HFileArchiver-8] backup.HFileArchiver(596): Deleted hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitWithoutPONR/75e2592cb2ac08bd9f617bb9d0a0c896 2023-05-10 04:53:54,354 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitWithoutPONR regions 2023-05-10 04:53:54,357 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=40, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-10 04:53:54,359 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitWithoutPONR from hbase:meta 2023-05-10 04:53:54,361 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitWithoutPONR' descriptor. 2023-05-10 04:53:54,363 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=40, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-10 04:53:54,363 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitWithoutPONR' from region states. 2023-05-10 04:53:54,363 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR,,1683694427650.6c70954228a0bbe97dfdc76861367c66.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1683694434363"}]},"ts":"9223372036854775807"} 2023-05-10 04:53:54,363 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR,,1683694429655.75e2592cb2ac08bd9f617bb9d0a0c896.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1683694434363"}]},"ts":"9223372036854775807"} 2023-05-10 04:53:54,363 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR,41,1683694429655.b90fd344aff14f0728a80c3d09539e54.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1683694434363"}]},"ts":"9223372036854775807"} 2023-05-10 04:53:54,365 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-05-10 04:53:54,365 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 6c70954228a0bbe97dfdc76861367c66, NAME => 'testSplitWithoutPONR,,1683694427650.6c70954228a0bbe97dfdc76861367c66.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 75e2592cb2ac08bd9f617bb9d0a0c896, NAME => 'testSplitWithoutPONR,,1683694429655.75e2592cb2ac08bd9f617bb9d0a0c896.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => b90fd344aff14f0728a80c3d09539e54, NAME => 'testSplitWithoutPONR,41,1683694429655.b90fd344aff14f0728a80c3d09539e54.', STARTKEY => '41', ENDKEY => ''}] 2023-05-10 04:53:54,365 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitWithoutPONR' as deleted. 2023-05-10 04:53:54,365 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1683694434365"}]},"ts":"9223372036854775807"} 2023-05-10 04:53:54,367 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitWithoutPONR state from META 2023-05-10 04:53:54,369 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=40, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-10 04:53:54,370 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR in 89 msec 2023-05-10 04:53:54,385 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(1227): Checking to see if procedure is done pid=40 2023-05-10 04:53:54,386 INFO [Listener at localhost/45007] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitWithoutPONR, procId: 40 completed 2023-05-10 04:53:54,400 INFO [Listener at localhost/45007] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitWithoutPONR Thread=398 (was 393) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-515825336_15 at /127.0.0.1:53066 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5d73dea0-99f3-3103-481a-a34861abc6a4/cluster_105d7649-9bd1-85f0-42ad-07d3b5dd11a5/dfs/data/data4/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-515825336_15 at /127.0.0.1:55428 [Waiting for operation #24] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-515825336_15 at /127.0.0.1:47324 [Waiting for operation #19] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-515825336_15 at /127.0.0.1:41522 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-515825336_15 at /127.0.0.1:50856 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4a1c5cf4-shared-pool-9 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-515825336_15 at /127.0.0.1:46346 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x248fdd40-shared-pool-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-515825336_15 at /127.0.0.1:47266 [Waiting for operation #20] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5d73dea0-99f3-3103-481a-a34861abc6a4/cluster_105d7649-9bd1-85f0-42ad-07d3b5dd11a5/dfs/data/data6/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5d73dea0-99f3-3103-481a-a34861abc6a4/cluster_105d7649-9bd1-85f0-42ad-07d3b5dd11a5/dfs/data/data5/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-515825336_15 at /127.0.0.1:52870 [Waiting for operation #16] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x248fdd40-shared-pool-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x4a1c5cf4-shared-pool-10 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-515825336_15 at /127.0.0.1:41450 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-515825336_15 at /127.0.0.1:47638 [Waiting for operation #16] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x248fdd40-metaLookup-shared--pool-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-515825336_15 at /127.0.0.1:53026 [Waiting for operation #16] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-515825336_15 at /127.0.0.1:41538 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5d73dea0-99f3-3103-481a-a34861abc6a4/cluster_105d7649-9bd1-85f0-42ad-07d3b5dd11a5/dfs/data/data3/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x248fdd40-shared-pool-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-515825336_15 at /127.0.0.1:47388 [Waiting for operation #28] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-515825336_15 at /127.0.0.1:46868 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x248fdd40-shared-pool-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x248fdd40-metaLookup-shared--pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-515825336_15 at /127.0.0.1:52854 [Waiting for operation #22] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-515825336_15 at /127.0.0.1:41440 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=848 (was 831) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=336 (was 348), ProcessCount=175 (was 175), AvailableMemoryMB=9725 (was 9728) 2023-05-10 04:53:54,418 INFO [Listener at localhost/45007] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testRollbackAndDoubleExecution Thread=398, OpenFileDescriptor=848, MaxFileDescriptor=60000, SystemLoadAverage=336, ProcessCount=175, AvailableMemoryMB=9724 2023-05-10 04:53:54,418 WARN [Listener at localhost/45007] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-05-10 04:53:54,422 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-05-10 04:53:54,422 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-05-10 04:53:54,424 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(1029): Stored pid=41, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-10 04:53:54,426 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_PRE_OPERATION 2023-05-10 04:53:54,427 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-05-10 04:53:54,429 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855 2023-05-10 04:53:54,429 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855 empty. 2023-05-10 04:53:54,430 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855 2023-05-10 04:53:54,430 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackAndDoubleExecution regions 2023-05-10 04:53:54,441 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testRollbackAndDoubleExecution/.tabledesc/.tableinfo.0000000001 2023-05-10 04:53:54,442 INFO [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(7675): creating {ENCODED => 4dcb62249031b560a1bc5d03e43ac855, NAME => 'testRollbackAndDoubleExecution,,1683694434423.4dcb62249031b560a1bc5d03e43ac855.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testRollbackAndDoubleExecution', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp 2023-05-10 04:53:54,451 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(866): Instantiated testRollbackAndDoubleExecution,,1683694434423.4dcb62249031b560a1bc5d03e43ac855.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-10 04:53:54,451 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1604): Closing 4dcb62249031b560a1bc5d03e43ac855, disabling compactions & flushes 2023-05-10 04:53:54,451 INFO [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1626): Closing region testRollbackAndDoubleExecution,,1683694434423.4dcb62249031b560a1bc5d03e43ac855. 2023-05-10 04:53:54,451 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRollbackAndDoubleExecution,,1683694434423.4dcb62249031b560a1bc5d03e43ac855. 2023-05-10 04:53:54,451 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1714): Acquired close lock on testRollbackAndDoubleExecution,,1683694434423.4dcb62249031b560a1bc5d03e43ac855. after waiting 0 ms 2023-05-10 04:53:54,451 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1724): Updates disabled for region testRollbackAndDoubleExecution,,1683694434423.4dcb62249031b560a1bc5d03e43ac855. 2023-05-10 04:53:54,451 INFO [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1838): Closed testRollbackAndDoubleExecution,,1683694434423.4dcb62249031b560a1bc5d03e43ac855. 2023-05-10 04:53:54,451 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1558): Region close journal for 4dcb62249031b560a1bc5d03e43ac855: 2023-05-10 04:53:54,454 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_ADD_TO_META 2023-05-10 04:53:54,455 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testRollbackAndDoubleExecution,,1683694434423.4dcb62249031b560a1bc5d03e43ac855.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1683694434454"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683694434454"}]},"ts":"1683694434454"} 2023-05-10 04:53:54,456 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-05-10 04:53:54,457 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-05-10 04:53:54,457 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683694434457"}]},"ts":"1683694434457"} 2023-05-10 04:53:54,458 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=ENABLING in hbase:meta 2023-05-10 04:53:54,461 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-05-10 04:53:54,462 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-05-10 04:53:54,463 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-05-10 04:53:54,463 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-05-10 04:53:54,463 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-05-10 04:53:54,463 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=42, ppid=41, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=4dcb62249031b560a1bc5d03e43ac855, ASSIGN}] 2023-05-10 04:53:54,464 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=42, ppid=41, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=4dcb62249031b560a1bc5d03e43ac855, ASSIGN 2023-05-10 04:53:54,465 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=42, ppid=41, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=4dcb62249031b560a1bc5d03e43ac855, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,37823,1683694419534; forceNewPlan=false, retain=false 2023-05-10 04:53:54,616 INFO [jenkins-hbase4:37903] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-05-10 04:53:54,617 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=42 updating hbase:meta row=4dcb62249031b560a1bc5d03e43ac855, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:54,617 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1683694434423.4dcb62249031b560a1bc5d03e43ac855.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1683694434617"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683694434617"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683694434617"}]},"ts":"1683694434617"} 2023-05-10 04:53:54,619 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=43, ppid=42, state=RUNNABLE; OpenRegionProcedure 4dcb62249031b560a1bc5d03e43ac855, server=jenkins-hbase4.apache.org,37823,1683694419534}] 2023-05-10 04:53:54,774 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testRollbackAndDoubleExecution,,1683694434423.4dcb62249031b560a1bc5d03e43ac855. 2023-05-10 04:53:54,774 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 4dcb62249031b560a1bc5d03e43ac855, NAME => 'testRollbackAndDoubleExecution,,1683694434423.4dcb62249031b560a1bc5d03e43ac855.', STARTKEY => '', ENDKEY => ''} 2023-05-10 04:53:54,775 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-05-10 04:53:54,775 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackAndDoubleExecution 4dcb62249031b560a1bc5d03e43ac855 2023-05-10 04:53:54,775 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testRollbackAndDoubleExecution,,1683694434423.4dcb62249031b560a1bc5d03e43ac855.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-10 04:53:54,775 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 4dcb62249031b560a1bc5d03e43ac855 2023-05-10 04:53:54,775 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 4dcb62249031b560a1bc5d03e43ac855 2023-05-10 04:53:54,776 INFO [StoreOpener-4dcb62249031b560a1bc5d03e43ac855-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 4dcb62249031b560a1bc5d03e43ac855 2023-05-10 04:53:54,778 DEBUG [StoreOpener-4dcb62249031b560a1bc5d03e43ac855-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf1 2023-05-10 04:53:54,778 DEBUG [StoreOpener-4dcb62249031b560a1bc5d03e43ac855-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf1 2023-05-10 04:53:54,778 INFO [StoreOpener-4dcb62249031b560a1bc5d03e43ac855-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 4dcb62249031b560a1bc5d03e43ac855 columnFamilyName cf1 2023-05-10 04:53:54,779 INFO [StoreOpener-4dcb62249031b560a1bc5d03e43ac855-1] regionserver.HStore(310): Store=4dcb62249031b560a1bc5d03e43ac855/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-10 04:53:54,779 INFO [StoreOpener-4dcb62249031b560a1bc5d03e43ac855-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 4dcb62249031b560a1bc5d03e43ac855 2023-05-10 04:53:54,780 DEBUG [StoreOpener-4dcb62249031b560a1bc5d03e43ac855-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf2 2023-05-10 04:53:54,780 DEBUG [StoreOpener-4dcb62249031b560a1bc5d03e43ac855-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf2 2023-05-10 04:53:54,781 INFO [StoreOpener-4dcb62249031b560a1bc5d03e43ac855-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 4dcb62249031b560a1bc5d03e43ac855 columnFamilyName cf2 2023-05-10 04:53:54,781 INFO [StoreOpener-4dcb62249031b560a1bc5d03e43ac855-1] regionserver.HStore(310): Store=4dcb62249031b560a1bc5d03e43ac855/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-10 04:53:54,782 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855 2023-05-10 04:53:54,782 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855 2023-05-10 04:53:54,784 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRollbackAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-05-10 04:53:54,785 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 4dcb62249031b560a1bc5d03e43ac855 2023-05-10 04:53:54,787 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-05-10 04:53:54,787 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 4dcb62249031b560a1bc5d03e43ac855; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72204075, jitterRate=0.07592456042766571}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-05-10 04:53:54,787 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 4dcb62249031b560a1bc5d03e43ac855: 2023-05-10 04:53:54,788 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2334): Post open deploy tasks for testRollbackAndDoubleExecution,,1683694434423.4dcb62249031b560a1bc5d03e43ac855., pid=43, masterSystemTime=1683694434771 2023-05-10 04:53:54,790 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2361): Finished post open deploy task for testRollbackAndDoubleExecution,,1683694434423.4dcb62249031b560a1bc5d03e43ac855. 2023-05-10 04:53:54,790 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testRollbackAndDoubleExecution,,1683694434423.4dcb62249031b560a1bc5d03e43ac855. 2023-05-10 04:53:54,790 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=42 updating hbase:meta row=4dcb62249031b560a1bc5d03e43ac855, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:54,791 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackAndDoubleExecution,,1683694434423.4dcb62249031b560a1bc5d03e43ac855.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1683694434790"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1683694434790"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1683694434790"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683694434790"}]},"ts":"1683694434790"} 2023-05-10 04:53:54,794 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=43, resume processing ppid=42 2023-05-10 04:53:54,794 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=43, ppid=42, state=SUCCESS; OpenRegionProcedure 4dcb62249031b560a1bc5d03e43ac855, server=jenkins-hbase4.apache.org,37823,1683694419534 in 173 msec 2023-05-10 04:53:54,796 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=42, resume processing ppid=41 2023-05-10 04:53:54,796 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=42, ppid=41, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=4dcb62249031b560a1bc5d03e43ac855, ASSIGN in 331 msec 2023-05-10 04:53:54,797 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-05-10 04:53:54,797 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683694434797"}]},"ts":"1683694434797"} 2023-05-10 04:53:54,798 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=ENABLED in hbase:meta 2023-05-10 04:53:54,805 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_POST_OPERATION 2023-05-10 04:53:54,807 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution in 382 msec 2023-05-10 04:53:54,934 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-10 04:53:54,936 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-05-10 04:53:54,936 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-05-10 04:53:54,936 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:53:54,936 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-05-10 04:53:54,936 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-05-10 04:53:54,936 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-10 04:53:54,937 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-10 04:53:54,938 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:53:54,938 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:53:54,938 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:53:54,938 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:53:54,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:53:54,938 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:54,938 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:53:54,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:53:54,938 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:53:54,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:53:54,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-10 04:53:54,939 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:54,939 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-10 04:53:54,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-10 04:53:54,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-10 04:53:54,939 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-10 04:53:54,939 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-10 04:53:54,939 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-10 04:53:54,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:53:54,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-10 04:53:54,939 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-05-10 04:53:54,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:53:54,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-10 04:53:54,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:53:54,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-10 04:53:54,939 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-05-10 04:53:54,940 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-05-10 04:53:54,940 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-05-10 04:53:54,940 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-05-10 04:53:54,940 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-05-10 04:53:54,940 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:53:54,940 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-05-10 04:53:54,940 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-05-10 04:53:54,940 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-05-10 04:53:54,940 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-05-10 04:53:54,940 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:53:54,940 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-05-10 04:53:54,940 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:53:54,940 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37995,1683694419499' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-05-10 04:53:54,940 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-05-10 04:53:54,940 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-05-10 04:53:54,940 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-05-10 04:53:54,940 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36665,1683694419452' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-05-10 04:53:54,940 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-10 04:53:54,941 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-10 04:53:54,941 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1683694434423.4dcb62249031b560a1bc5d03e43ac855. 2023-05-10 04:53:54,941 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1683694434423.4dcb62249031b560a1bc5d03e43ac855. started... 2023-05-10 04:53:54,942 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 4dcb62249031b560a1bc5d03e43ac855 2/2 column families, dataSize=120 B heapSize=928 B 2023-05-10 04:53:54,942 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-10 04:53:54,943 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-10 04:53:54,943 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-10 04:53:54,943 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:53:54,943 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-10 04:53:54,943 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:53:54,952 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/.tmp/cf1/be588b1c116843cb87e38baf82c3a2db 2023-05-10 04:53:54,967 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/.tmp/cf2/d22e75e4a5124b3299106ffa2fc21709 2023-05-10 04:53:54,971 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/.tmp/cf1/be588b1c116843cb87e38baf82c3a2db as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf1/be588b1c116843cb87e38baf82c3a2db 2023-05-10 04:53:54,977 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf1/be588b1c116843cb87e38baf82c3a2db, entries=2, sequenceid=6, filesize=4.8 K 2023-05-10 04:53:54,978 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/.tmp/cf2/d22e75e4a5124b3299106ffa2fc21709 as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf2/d22e75e4a5124b3299106ffa2fc21709 2023-05-10 04:53:54,983 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf2/d22e75e4a5124b3299106ffa2fc21709, entries=2, sequenceid=6, filesize=4.8 K 2023-05-10 04:53:54,984 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 4dcb62249031b560a1bc5d03e43ac855 in 42ms, sequenceid=6, compaction requested=false 2023-05-10 04:53:54,984 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testRollbackAndDoubleExecution' 2023-05-10 04:53:54,985 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 4dcb62249031b560a1bc5d03e43ac855: 2023-05-10 04:53:54,985 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1683694434423.4dcb62249031b560a1bc5d03e43ac855. 2023-05-10 04:53:54,985 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-10 04:53:54,985 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-10 04:53:54,985 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:54,985 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-05-10 04:53:54,985 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37823,1683694419534' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-05-10 04:53:54,988 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:54,988 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-10 04:53:54,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:54,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:53:54,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:53:54,989 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-10 04:53:54,989 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:53:54,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:53:54,989 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:53:54,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-10 04:53:54,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:54,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:54,990 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:54,991 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:53:54,991 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37823,1683694419534' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-05-10 04:53:54,991 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-05-10 04:53:54,991 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@64fb7fde[Count = 0] remaining members to acquire global barrier 2023-05-10 04:53:54,991 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-10 04:53:54,992 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-10 04:53:54,993 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-10 04:53:54,992 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-10 04:53:54,993 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-10 04:53:54,993 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-10 04:53:54,993 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:54,993 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-10 04:53:54,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-10 04:53:54,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-10 04:53:54,993 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-05-10 04:53:54,993 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-10 04:53:54,993 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-05-10 04:53:54,993 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-05-10 04:53:54,993 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,37995,1683694419499' in zk 2023-05-10 04:53:54,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-10 04:53:54,993 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-05-10 04:53:54,993 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-05-10 04:53:54,993 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,37823,1683694419534' in zk 2023-05-10 04:53:54,994 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-05-10 04:53:54,994 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,36665,1683694419452' in zk 2023-05-10 04:53:54,995 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-05-10 04:53:54,995 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:53:54,995 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-05-10 04:53:54,996 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-05-10 04:53:54,996 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:54,996 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:53:54,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:54,996 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-05-10 04:53:54,996 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-05-10 04:53:54,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:53:54,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:53:54,996 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:53:54,997 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-05-10 04:53:54,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:53:54,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:53:54,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-10 04:53:54,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:54,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:54,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:54,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:53:54,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-10 04:53:54,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:54,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:55,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:55,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,37823,1683694419534': 2023-05-10 04:53:55,000 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37823,1683694419534' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-05-10 04:53:55,000 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-05-10 04:53:55,000 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-10 04:53:55,000 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-10 04:53:55,001 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-05-10 04:53:55,001 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-10 04:53:55,003 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-10 04:53:55,003 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-10 04:53:55,003 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-10 04:53:55,004 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-10 04:53:55,004 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:53:55,003 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-10 04:53:55,003 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:53:55,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-10 04:53:55,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:53:55,003 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-10 04:53:55,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:53:55,004 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-10 04:53:55,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-10 04:53:55,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-10 04:53:55,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-10 04:53:55,004 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:55,004 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:53:55,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:53:55,005 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:53:55,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:53:55,005 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:53:55,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:53:55,005 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:53:55,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:53:55,005 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:55,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-10 04:53:55,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-10 04:53:55,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-10 04:53:55,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-10 04:53:55,006 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:55,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:53:55,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-10 04:53:55,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:55,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:55,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:55,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:53:55,008 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:55,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-10 04:53:55,008 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:55,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:55,008 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:55,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:55,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:55,012 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:55,012 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:53:55,012 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:53:55,012 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:53:55,012 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-10 04:53:55,012 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:53:55,012 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:53:55,012 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:53:55,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:53:55,012 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:53:55,012 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:53:55,012 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:53:55,012 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:53:55,012 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-05-10 04:53:55,013 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-10 04:53:55,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:53:55,012 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:55,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:53:55,013 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-05-10 04:53:55,013 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:53:55,013 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:55,013 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:53:55,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:53:55,013 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-10 04:53:55,013 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:53:55,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:53:55,013 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-10 04:53:55,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:53:55,013 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:55,014 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-10 04:53:55,014 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:55,014 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:55,014 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-10 04:53:55,014 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-10 04:53:55,114 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-10 04:53:55,115 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-10 04:53:55,131 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-10 04:53:55,133 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-05-10 04:53:55,133 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-05-10 04:53:55,133 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-05-10 04:53:55,133 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:53:55,133 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-05-10 04:53:55,133 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-05-10 04:53:55,134 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-10 04:53:55,134 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-10 04:53:55,135 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:53:55,135 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:55,135 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:53:55,135 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:53:55,135 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:53:55,135 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:53:55,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:53:55,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:53:55,135 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:53:55,136 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:53:55,137 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:55,137 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-10 04:53:55,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-10 04:53:55,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-10 04:53:55,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-10 04:53:55,138 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-10 04:53:55,138 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-10 04:53:55,138 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-10 04:53:55,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:53:55,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-10 04:53:55,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:53:55,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-10 04:53:55,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:53:55,138 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-10 04:53:55,138 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-05-10 04:53:55,138 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-05-10 04:53:55,138 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-05-10 04:53:55,139 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-05-10 04:53:55,139 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-05-10 04:53:55,139 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-05-10 04:53:55,139 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-05-10 04:53:55,139 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-05-10 04:53:55,139 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-05-10 04:53:55,139 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-05-10 04:53:55,139 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:53:55,139 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-05-10 04:53:55,139 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-05-10 04:53:55,139 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-05-10 04:53:55,139 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:53:55,139 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:53:55,139 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-05-10 04:53:55,139 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-10 04:53:55,139 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-10 04:53:55,139 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1683694434423.4dcb62249031b560a1bc5d03e43ac855. 2023-05-10 04:53:55,139 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-05-10 04:53:55,139 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1683694434423.4dcb62249031b560a1bc5d03e43ac855. started... 2023-05-10 04:53:55,140 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-05-10 04:53:55,139 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-05-10 04:53:55,140 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37995,1683694419499' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-05-10 04:53:55,140 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36665,1683694419452' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-05-10 04:53:55,140 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 4dcb62249031b560a1bc5d03e43ac855 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-10 04:53:55,141 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-10 04:53:55,143 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-10 04:53:55,143 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-10 04:53:55,143 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:53:55,144 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-10 04:53:55,144 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:53:55,157 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/.tmp/cf1/fc0d292a16e34289b813715b5a9e56bf 2023-05-10 04:53:55,576 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/.tmp/cf2/11c6828a4683438fba9bf4cd76081c5d 2023-05-10 04:53:55,581 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/.tmp/cf1/fc0d292a16e34289b813715b5a9e56bf as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf1/fc0d292a16e34289b813715b5a9e56bf 2023-05-10 04:53:55,585 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf1/fc0d292a16e34289b813715b5a9e56bf, entries=10, sequenceid=19, filesize=5.0 K 2023-05-10 04:53:55,586 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/.tmp/cf2/11c6828a4683438fba9bf4cd76081c5d as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf2/11c6828a4683438fba9bf4cd76081c5d 2023-05-10 04:53:55,590 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf2/11c6828a4683438fba9bf4cd76081c5d, entries=10, sequenceid=19, filesize=5.0 K 2023-05-10 04:53:55,591 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 4dcb62249031b560a1bc5d03e43ac855 in 451ms, sequenceid=19, compaction requested=false 2023-05-10 04:53:55,591 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 4dcb62249031b560a1bc5d03e43ac855: 2023-05-10 04:53:55,591 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1683694434423.4dcb62249031b560a1bc5d03e43ac855. 2023-05-10 04:53:55,591 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-10 04:53:55,592 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-10 04:53:55,592 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:55,592 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-05-10 04:53:55,592 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37823,1683694419534' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-05-10 04:53:55,594 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:55,594 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-10 04:53:55,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:55,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:53:55,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:53:55,594 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-10 04:53:55,594 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:53:55,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:53:55,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:53:55,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-10 04:53:55,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:55,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:55,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:55,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:53:55,597 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37823,1683694419534' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-05-10 04:53:55,597 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@549c26b7[Count = 0] remaining members to acquire global barrier 2023-05-10 04:53:55,597 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-05-10 04:53:55,597 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-10 04:53:55,598 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-10 04:53:55,598 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-10 04:53:55,598 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-10 04:53:55,598 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-10 04:53:55,598 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-10 04:53:55,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-10 04:53:55,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-10 04:53:55,598 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-05-10 04:53:55,598 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-05-10 04:53:55,598 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-10 04:53:55,598 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-05-10 04:53:55,598 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:55,598 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-05-10 04:53:55,599 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-10 04:53:55,599 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,37995,1683694419499' in zk 2023-05-10 04:53:55,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-10 04:53:55,599 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-05-10 04:53:55,599 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,36665,1683694419452' in zk 2023-05-10 04:53:55,599 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-05-10 04:53:55,599 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,37823,1683694419534' in zk 2023-05-10 04:53:55,600 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-05-10 04:53:55,600 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:53:55,600 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-05-10 04:53:55,604 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-05-10 04:53:55,604 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:55,604 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:53:55,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:55,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:53:55,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:53:55,604 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-05-10 04:53:55,604 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-05-10 04:53:55,605 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:53:55,605 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-05-10 04:53:55,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:53:55,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:53:55,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-10 04:53:55,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:55,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:55,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:55,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:53:55,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-10 04:53:55,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:55,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:55,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:55,608 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,37823,1683694419534': 2023-05-10 04:53:55,608 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37823,1683694419534' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-05-10 04:53:55,608 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-05-10 04:53:55,608 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-10 04:53:55,608 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-10 04:53:55,608 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-05-10 04:53:55,608 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-10 04:53:55,609 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-10 04:53:55,610 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-10 04:53:55,610 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:53:55,610 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-10 04:53:55,610 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:53:55,610 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-10 04:53:55,610 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-10 04:53:55,610 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-10 04:53:55,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-10 04:53:55,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:53:55,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:53:55,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-10 04:53:55,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-10 04:53:55,610 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:53:55,610 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-10 04:53:55,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-10 04:53:55,610 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:55,610 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:53:55,610 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:53:55,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:53:55,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:53:55,611 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:53:55,610 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:53:55,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-10 04:53:55,611 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:55,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:53:55,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-10 04:53:55,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-10 04:53:55,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-10 04:53:55,611 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:55,611 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:53:55,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-10 04:53:55,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:55,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:55,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:55,613 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:55,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:53:55,613 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:55,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-10 04:53:55,613 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:55,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:55,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:55,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:55,616 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:55,617 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-10 04:53:55,617 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:53:55,617 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:55,617 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:53:55,617 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:53:55,617 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:55,617 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-05-10 04:53:55,617 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-10 04:53:55,617 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-10 04:53:55,617 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:55,617 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:53:55,617 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-10 04:53:55,617 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:53:55,617 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:53:55,617 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:53:55,617 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-05-10 04:53:55,617 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:53:55,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:53:55,617 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-10 04:53:55,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:53:55,617 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:55,617 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:53:55,617 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:53:55,618 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:53:55,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:53:55,618 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:53:55,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:53:55,618 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:55,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:53:55,618 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-10 04:53:55,618 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-10 04:53:55,618 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:53:55,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:53:55,718 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-10 04:53:55,719 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-10 04:53:55,735 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-10 04:53:55,737 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-05-10 04:53:55,737 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-05-10 04:53:55,737 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-05-10 04:53:55,737 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:53:55,738 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-05-10 04:53:55,738 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-05-10 04:53:55,738 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-10 04:53:55,738 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-10 04:53:55,740 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:53:55,740 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:53:55,740 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:53:55,740 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:55,740 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:53:55,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:53:55,740 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:53:55,740 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:53:55,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:53:55,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:53:55,740 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:55,741 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-10 04:53:55,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-10 04:53:55,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-10 04:53:55,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-10 04:53:55,741 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-10 04:53:55,741 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-10 04:53:55,741 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-10 04:53:55,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:53:55,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-10 04:53:55,741 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-05-10 04:53:55,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:53:55,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-10 04:53:55,742 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-05-10 04:53:55,742 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-05-10 04:53:55,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:53:55,742 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-05-10 04:53:55,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-10 04:53:55,742 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-05-10 04:53:55,742 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-05-10 04:53:55,742 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-05-10 04:53:55,742 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-05-10 04:53:55,742 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:53:55,742 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-05-10 04:53:55,742 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-05-10 04:53:55,742 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-05-10 04:53:55,742 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:53:55,742 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-05-10 04:53:55,742 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-05-10 04:53:55,743 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-05-10 04:53:55,743 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-05-10 04:53:55,743 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:53:55,743 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37995,1683694419499' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-05-10 04:53:55,743 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-05-10 04:53:55,743 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-05-10 04:53:55,743 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36665,1683694419452' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-05-10 04:53:55,744 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-10 04:53:55,744 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-10 04:53:55,744 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1683694434423.4dcb62249031b560a1bc5d03e43ac855. 2023-05-10 04:53:55,744 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1683694434423.4dcb62249031b560a1bc5d03e43ac855. started... 2023-05-10 04:53:55,744 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 4dcb62249031b560a1bc5d03e43ac855 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-10 04:53:55,746 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-10 04:53:55,746 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-10 04:53:55,747 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-10 04:53:55,747 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-10 04:53:55,747 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:53:55,747 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:53:55,754 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/.tmp/cf1/439486fbe6944c55ac7e2e681288c043 2023-05-10 04:53:55,772 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/.tmp/cf2/3d1fe957545445a4befe08b6b30a80dc 2023-05-10 04:53:55,778 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/.tmp/cf1/439486fbe6944c55ac7e2e681288c043 as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf1/439486fbe6944c55ac7e2e681288c043 2023-05-10 04:53:55,782 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf1/439486fbe6944c55ac7e2e681288c043, entries=10, sequenceid=32, filesize=5.0 K 2023-05-10 04:53:55,783 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/.tmp/cf2/3d1fe957545445a4befe08b6b30a80dc as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf2/3d1fe957545445a4befe08b6b30a80dc 2023-05-10 04:53:55,787 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf2/3d1fe957545445a4befe08b6b30a80dc, entries=10, sequenceid=32, filesize=5.0 K 2023-05-10 04:53:55,788 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 4dcb62249031b560a1bc5d03e43ac855 in 44ms, sequenceid=32, compaction requested=true 2023-05-10 04:53:55,788 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 4dcb62249031b560a1bc5d03e43ac855: 2023-05-10 04:53:55,789 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1683694434423.4dcb62249031b560a1bc5d03e43ac855. 2023-05-10 04:53:55,789 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-10 04:53:55,789 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-10 04:53:55,789 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:55,789 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-05-10 04:53:55,789 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37823,1683694419534' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-05-10 04:53:55,791 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-10 04:53:55,791 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:55,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:55,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:53:55,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:53:55,791 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-10 04:53:55,791 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:53:55,791 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:53:55,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:53:55,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-10 04:53:55,792 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:55,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:55,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:55,793 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:53:55,793 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37823,1683694419534' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-05-10 04:53:55,793 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@43c47188[Count = 0] remaining members to acquire global barrier 2023-05-10 04:53:55,793 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-05-10 04:53:55,793 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-10 04:53:55,795 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-10 04:53:55,795 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-10 04:53:55,795 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-10 04:53:55,795 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-10 04:53:55,795 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-10 04:53:55,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-10 04:53:55,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-10 04:53:55,795 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-05-10 04:53:55,795 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:55,795 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-10 04:53:55,795 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-10 04:53:55,795 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-05-10 04:53:55,795 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-05-10 04:53:55,796 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,36665,1683694419452' in zk 2023-05-10 04:53:55,795 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-10 04:53:55,796 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-05-10 04:53:55,796 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-05-10 04:53:55,796 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-05-10 04:53:55,796 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,37823,1683694419534' in zk 2023-05-10 04:53:55,796 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,37995,1683694419499' in zk 2023-05-10 04:53:55,797 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-05-10 04:53:55,797 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:53:55,798 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-05-10 04:53:55,801 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-05-10 04:53:55,801 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:53:55,801 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-05-10 04:53:55,801 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:55,801 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:53:55,801 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-05-10 04:53:55,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:55,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:53:55,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:53:55,802 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-05-10 04:53:55,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:53:55,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:53:55,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-10 04:53:55,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:55,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:55,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:55,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:53:55,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-10 04:53:55,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:55,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:55,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:55,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,37823,1683694419534': 2023-05-10 04:53:55,805 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37823,1683694419534' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-05-10 04:53:55,805 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-05-10 04:53:55,805 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-10 04:53:55,805 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-10 04:53:55,805 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-05-10 04:53:55,805 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-10 04:53:55,807 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-10 04:53:55,807 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-10 04:53:55,807 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-10 04:53:55,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-10 04:53:55,807 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-10 04:53:55,807 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:53:55,808 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-10 04:53:55,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-10 04:53:55,807 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-10 04:53:55,808 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:53:55,808 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:55,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:53:55,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-10 04:53:55,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:53:55,808 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-10 04:53:55,808 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:53:55,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-10 04:53:55,808 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:53:55,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:53:55,808 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:53:55,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:53:55,808 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:55,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-10 04:53:55,809 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:53:55,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:53:55,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:53:55,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-10 04:53:55,809 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:55,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-10 04:53:55,809 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-10 04:53:55,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:53:55,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-10 04:53:55,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:55,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:55,811 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:55,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:55,811 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:55,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:53:55,812 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:55,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-10 04:53:55,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:55,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:55,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:55,818 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:55,818 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-10 04:53:55,818 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:55,818 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:53:55,818 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:53:55,818 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:55,818 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:53:55,819 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-10 04:53:55,818 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:53:55,818 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:53:55,818 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-05-10 04:53:55,819 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-10 04:53:55,818 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:53:55,818 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:53:55,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:53:55,818 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:53:55,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:53:55,819 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:55,819 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:53:55,819 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:53:55,819 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-05-10 04:53:55,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:53:55,819 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-10 04:53:55,819 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-10 04:53:55,819 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:55,820 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:55,820 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-10 04:53:55,820 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-10 04:53:55,820 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:53:55,820 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:53:55,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:53:55,820 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:53:55,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:53:55,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:53:55,920 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-10 04:53:55,921 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-10 04:53:55,938 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-10 04:53:55,940 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-05-10 04:53:55,940 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-05-10 04:53:55,940 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-05-10 04:53:55,940 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:53:55,940 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-05-10 04:53:55,940 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-05-10 04:53:55,940 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-10 04:53:55,940 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-10 04:53:55,942 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:53:55,942 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:53:55,942 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:55,942 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:53:55,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:53:55,942 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:53:55,942 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:53:55,942 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:53:55,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:53:55,942 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:55,942 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-10 04:53:55,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:53:55,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-10 04:53:55,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-10 04:53:55,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-10 04:53:55,943 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-10 04:53:55,943 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-10 04:53:55,943 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-10 04:53:55,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:53:55,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:53:55,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-10 04:53:55,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:53:55,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-10 04:53:55,943 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-05-10 04:53:55,943 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-05-10 04:53:55,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-10 04:53:55,944 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-05-10 04:53:55,944 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-05-10 04:53:55,944 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-05-10 04:53:55,944 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-05-10 04:53:55,944 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-05-10 04:53:55,944 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-05-10 04:53:55,944 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-05-10 04:53:55,944 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-05-10 04:53:55,944 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:53:55,944 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:53:55,944 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-05-10 04:53:55,944 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-05-10 04:53:55,944 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-05-10 04:53:55,944 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-05-10 04:53:55,944 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37995,1683694419499' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-05-10 04:53:55,944 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-05-10 04:53:55,944 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:53:55,944 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-05-10 04:53:55,944 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-05-10 04:53:55,944 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36665,1683694419452' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-05-10 04:53:55,945 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-10 04:53:55,945 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-10 04:53:55,945 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1683694434423.4dcb62249031b560a1bc5d03e43ac855. 2023-05-10 04:53:55,945 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1683694434423.4dcb62249031b560a1bc5d03e43ac855. started... 2023-05-10 04:53:55,945 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 4dcb62249031b560a1bc5d03e43ac855 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-10 04:53:55,946 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-10 04:53:55,948 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-10 04:53:55,948 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-10 04:53:55,948 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:53:55,949 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-10 04:53:55,949 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:53:55,958 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/.tmp/cf1/8df5350bb6b84fedabe8af09648762f1 2023-05-10 04:53:55,973 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/.tmp/cf2/50823f0b97364b55999bb34d2161b4b9 2023-05-10 04:53:55,979 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/.tmp/cf1/8df5350bb6b84fedabe8af09648762f1 as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf1/8df5350bb6b84fedabe8af09648762f1 2023-05-10 04:53:55,983 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf1/8df5350bb6b84fedabe8af09648762f1, entries=10, sequenceid=45, filesize=5.0 K 2023-05-10 04:53:55,984 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/.tmp/cf2/50823f0b97364b55999bb34d2161b4b9 as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf2/50823f0b97364b55999bb34d2161b4b9 2023-05-10 04:53:55,989 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf2/50823f0b97364b55999bb34d2161b4b9, entries=10, sequenceid=45, filesize=5.0 K 2023-05-10 04:53:55,990 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 4dcb62249031b560a1bc5d03e43ac855 in 45ms, sequenceid=45, compaction requested=true 2023-05-10 04:53:55,990 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 4dcb62249031b560a1bc5d03e43ac855: 2023-05-10 04:53:55,990 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1683694434423.4dcb62249031b560a1bc5d03e43ac855. 2023-05-10 04:53:55,990 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-10 04:53:55,990 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-10 04:53:55,990 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:55,990 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-05-10 04:53:55,991 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37823,1683694419534' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-05-10 04:53:55,992 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:55,992 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-10 04:53:55,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:55,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:53:55,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:53:55,992 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-10 04:53:55,992 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:53:55,992 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:53:55,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:53:55,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-10 04:53:55,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:55,993 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:55,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:55,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:53:55,994 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37823,1683694419534' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-05-10 04:53:55,994 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@77ffcc26[Count = 0] remaining members to acquire global barrier 2023-05-10 04:53:55,994 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-05-10 04:53:55,994 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-10 04:53:55,996 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-10 04:53:55,996 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-10 04:53:55,996 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-10 04:53:55,996 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-10 04:53:55,996 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-10 04:53:55,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-10 04:53:55,996 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:55,996 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-05-10 04:53:55,996 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-05-10 04:53:55,996 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-10 04:53:55,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-10 04:53:55,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-10 04:53:55,996 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,37995,1683694419499' in zk 2023-05-10 04:53:55,996 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-05-10 04:53:55,996 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-10 04:53:55,996 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-05-10 04:53:55,996 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-05-10 04:53:55,997 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-05-10 04:53:55,997 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,36665,1683694419452' in zk 2023-05-10 04:53:55,997 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,37823,1683694419534' in zk 2023-05-10 04:53:55,999 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-05-10 04:53:55,999 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-05-10 04:53:55,999 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:53:55,999 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:53:55,999 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-05-10 04:53:55,999 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-05-10 04:53:56,005 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-05-10 04:53:56,005 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:56,005 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:53:56,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:56,005 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-05-10 04:53:56,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:53:56,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:53:56,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:53:56,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:53:56,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-10 04:53:56,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:56,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:56,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:56,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:53:56,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-10 04:53:56,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:56,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:56,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:56,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,37823,1683694419534': 2023-05-10 04:53:56,009 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37823,1683694419534' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-05-10 04:53:56,009 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-05-10 04:53:56,009 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-10 04:53:56,009 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-10 04:53:56,009 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-05-10 04:53:56,009 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-10 04:53:56,010 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-10 04:53:56,011 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-10 04:53:56,011 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-10 04:53:56,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-10 04:53:56,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-10 04:53:56,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:53:56,011 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-10 04:53:56,011 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:53:56,011 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:53:56,011 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-10 04:53:56,011 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-10 04:53:56,011 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-10 04:53:56,011 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:56,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:53:56,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-10 04:53:56,011 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:53:56,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-10 04:53:56,011 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:53:56,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:53:56,011 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:56,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:53:56,012 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:53:56,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-10 04:53:56,012 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:53:56,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:53:56,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:53:56,012 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:56,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-10 04:53:56,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-10 04:53:56,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-10 04:53:56,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:53:56,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-10 04:53:56,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:56,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:56,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:56,014 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:56,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:53:56,014 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:56,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-10 04:53:56,014 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:56,014 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:56,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:56,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:56,017 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:56,017 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:53:56,017 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:53:56,017 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:53:56,017 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:53:56,017 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:53:56,017 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:53:56,017 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:53:56,017 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-10 04:53:56,017 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:53:56,017 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:53:56,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:53:56,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:53:56,017 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-05-10 04:53:56,017 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-10 04:53:56,017 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:53:56,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:53:56,018 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:53:56,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:53:56,017 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:56,018 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:53:56,018 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:56,018 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-05-10 04:53:56,018 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-10 04:53:56,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:53:56,018 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:53:56,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:53:56,018 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:56,018 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-10 04:53:56,018 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-10 04:53:56,018 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:56,018 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:56,018 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-10 04:53:56,018 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-10 04:53:56,118 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-10 04:53:56,119 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-10 04:53:56,135 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-10 04:53:56,137 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-05-10 04:53:56,137 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-05-10 04:53:56,137 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-05-10 04:53:56,137 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:53:56,138 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-05-10 04:53:56,138 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-05-10 04:53:56,138 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-10 04:53:56,138 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-10 04:53:56,140 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:53:56,140 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:53:56,140 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:53:56,140 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:53:56,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:53:56,140 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:53:56,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:53:56,140 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:53:56,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:53:56,140 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:56,140 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-10 04:53:56,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-10 04:53:56,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-10 04:53:56,141 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-10 04:53:56,141 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-10 04:53:56,147 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-10 04:53:56,147 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:56,147 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-10 04:53:56,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:53:56,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-10 04:53:56,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:53:56,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-10 04:53:56,147 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-05-10 04:53:56,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:53:56,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-10 04:53:56,147 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-05-10 04:53:56,180 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-05-10 04:53:56,181 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-05-10 04:53:56,181 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-05-10 04:53:56,181 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-05-10 04:53:56,181 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-05-10 04:53:56,181 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:53:56,181 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:53:56,181 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-05-10 04:53:56,181 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-05-10 04:53:56,181 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36665,1683694419452' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-05-10 04:53:56,181 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-05-10 04:53:56,182 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-05-10 04:53:56,182 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37995,1683694419499' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-05-10 04:53:56,182 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-05-10 04:53:56,182 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-05-10 04:53:56,182 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:53:56,182 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-05-10 04:53:56,182 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-10 04:53:56,182 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-10 04:53:56,183 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1683694434423.4dcb62249031b560a1bc5d03e43ac855. 2023-05-10 04:53:56,183 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1683694434423.4dcb62249031b560a1bc5d03e43ac855. started... 2023-05-10 04:53:56,183 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 4dcb62249031b560a1bc5d03e43ac855 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-10 04:53:56,186 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-10 04:53:56,186 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-10 04:53:56,186 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-10 04:53:56,186 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-10 04:53:56,186 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:53:56,186 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:53:56,194 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/.tmp/cf1/b348d048ffd647f0b322ec445c520c0e 2023-05-10 04:53:56,206 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/.tmp/cf2/d89cd6883f824b51afd167aaa88a54be 2023-05-10 04:53:56,210 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/.tmp/cf1/b348d048ffd647f0b322ec445c520c0e as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf1/b348d048ffd647f0b322ec445c520c0e 2023-05-10 04:53:56,215 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf1/b348d048ffd647f0b322ec445c520c0e, entries=10, sequenceid=58, filesize=5.0 K 2023-05-10 04:53:56,215 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/.tmp/cf2/d89cd6883f824b51afd167aaa88a54be as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf2/d89cd6883f824b51afd167aaa88a54be 2023-05-10 04:53:56,219 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf2/d89cd6883f824b51afd167aaa88a54be, entries=10, sequenceid=58, filesize=5.0 K 2023-05-10 04:53:56,220 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 4dcb62249031b560a1bc5d03e43ac855 in 37ms, sequenceid=58, compaction requested=true 2023-05-10 04:53:56,220 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 4dcb62249031b560a1bc5d03e43ac855: 2023-05-10 04:53:56,220 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1683694434423.4dcb62249031b560a1bc5d03e43ac855. 2023-05-10 04:53:56,220 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-10 04:53:56,220 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-10 04:53:56,220 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:56,220 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-05-10 04:53:56,220 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37823,1683694419534' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-05-10 04:53:56,222 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:56,222 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-10 04:53:56,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:56,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:53:56,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:53:56,222 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-10 04:53:56,222 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:53:56,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:53:56,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:53:56,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-10 04:53:56,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:56,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:56,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:56,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:53:56,224 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37823,1683694419534' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-05-10 04:53:56,224 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@48c54888[Count = 0] remaining members to acquire global barrier 2023-05-10 04:53:56,224 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-05-10 04:53:56,224 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-10 04:53:56,226 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-10 04:53:56,226 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-10 04:53:56,226 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-10 04:53:56,226 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-10 04:53:56,226 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-10 04:53:56,226 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:56,226 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-10 04:53:56,226 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-10 04:53:56,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-10 04:53:56,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-10 04:53:56,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-10 04:53:56,226 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-05-10 04:53:56,226 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-05-10 04:53:56,226 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,37823,1683694419534' in zk 2023-05-10 04:53:56,226 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-05-10 04:53:56,226 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-05-10 04:53:56,226 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-05-10 04:53:56,226 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-05-10 04:53:56,226 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,36665,1683694419452' in zk 2023-05-10 04:53:56,226 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,37995,1683694419499' in zk 2023-05-10 04:53:56,228 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:56,228 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-05-10 04:53:56,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:56,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:53:56,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:53:56,228 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:53:56,228 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-05-10 04:53:56,232 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-05-10 04:53:56,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:53:56,232 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-05-10 04:53:56,232 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:53:56,232 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:53:56,232 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-05-10 04:53:56,233 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-05-10 04:53:56,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:53:56,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-10 04:53:56,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:56,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:56,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:56,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:53:56,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-10 04:53:56,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:56,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:56,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:56,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,37823,1683694419534': 2023-05-10 04:53:56,235 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37823,1683694419534' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-05-10 04:53:56,235 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-05-10 04:53:56,235 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-10 04:53:56,235 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-10 04:53:56,235 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-05-10 04:53:56,235 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-10 04:53:56,237 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-10 04:53:56,237 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-10 04:53:56,237 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-10 04:53:56,237 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:53:56,237 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:53:56,237 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-10 04:53:56,237 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-10 04:53:56,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-10 04:53:56,237 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-10 04:53:56,237 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-10 04:53:56,237 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:53:56,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-10 04:53:56,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-10 04:53:56,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-10 04:53:56,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:53:56,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:53:56,237 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:53:56,237 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:56,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:53:56,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:53:56,238 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:53:56,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:53:56,238 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:56,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-10 04:53:56,238 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:53:56,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:53:56,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-10 04:53:56,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-10 04:53:56,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-10 04:53:56,238 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:56,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:53:56,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-10 04:53:56,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:56,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:56,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:56,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:53:56,240 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:56,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-10 04:53:56,240 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:56,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:56,241 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:56,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:56,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:56,243 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:53:56,243 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:53:56,243 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:53:56,243 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:53:56,243 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:53:56,243 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:56,243 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:53:56,243 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:53:56,243 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:53:56,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:53:56,243 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:53:56,243 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:53:56,243 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-05-10 04:53:56,244 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-10 04:53:56,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:53:56,243 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-10 04:53:56,244 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:53:56,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:53:56,244 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:56,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:53:56,244 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:53:56,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:53:56,244 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:56,244 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-05-10 04:53:56,244 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:53:56,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:53:56,244 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-10 04:53:56,244 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-10 04:53:56,244 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:56,244 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-10 04:53:56,245 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:56,245 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:56,245 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-10 04:53:56,245 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-10 04:53:56,345 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-10 04:53:56,346 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-10 04:53:56,360 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-10 04:53:56,362 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-05-10 04:53:56,362 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-05-10 04:53:56,362 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-05-10 04:53:56,362 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:53:56,362 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-05-10 04:53:56,362 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-05-10 04:53:56,363 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-10 04:53:56,363 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-10 04:53:56,365 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:53:56,365 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:53:56,365 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:53:56,365 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:53:56,365 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:56,365 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:53:56,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:53:56,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:53:56,365 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:53:56,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:53:56,365 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:56,365 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-10 04:53:56,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-10 04:53:56,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-10 04:53:56,365 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-10 04:53:56,366 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-10 04:53:56,366 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-10 04:53:56,366 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-10 04:53:56,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:53:56,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-10 04:53:56,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:53:56,366 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-05-10 04:53:56,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:53:56,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-10 04:53:56,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-10 04:53:56,366 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-05-10 04:53:56,366 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-05-10 04:53:56,367 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-05-10 04:53:56,367 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-05-10 04:53:56,367 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-05-10 04:53:56,367 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:53:56,367 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-05-10 04:53:56,367 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-05-10 04:53:56,367 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-05-10 04:53:56,367 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-05-10 04:53:56,367 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37995,1683694419499' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-05-10 04:53:56,367 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-05-10 04:53:56,367 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:53:56,367 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-05-10 04:53:56,367 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-05-10 04:53:56,367 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-05-10 04:53:56,367 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-05-10 04:53:56,367 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:53:56,367 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-10 04:53:56,367 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1683694434423.4dcb62249031b560a1bc5d03e43ac855. 2023-05-10 04:53:56,367 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-05-10 04:53:56,367 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1683694434423.4dcb62249031b560a1bc5d03e43ac855. started... 2023-05-10 04:53:56,367 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-05-10 04:53:56,367 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36665,1683694419452' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-05-10 04:53:56,367 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-10 04:53:56,368 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 4dcb62249031b560a1bc5d03e43ac855 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-10 04:53:56,369 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-10 04:53:56,370 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-10 04:53:56,370 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-10 04:53:56,370 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:53:56,370 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-10 04:53:56,370 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:53:56,380 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/.tmp/cf1/118eb6964a944d8bb1efaa8c1933daa3 2023-05-10 04:53:56,392 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/.tmp/cf2/c543f3f56d754808a11f335a526021df 2023-05-10 04:53:56,397 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/.tmp/cf1/118eb6964a944d8bb1efaa8c1933daa3 as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf1/118eb6964a944d8bb1efaa8c1933daa3 2023-05-10 04:53:56,400 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf1/118eb6964a944d8bb1efaa8c1933daa3, entries=10, sequenceid=71, filesize=5.0 K 2023-05-10 04:53:56,401 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/.tmp/cf2/c543f3f56d754808a11f335a526021df as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf2/c543f3f56d754808a11f335a526021df 2023-05-10 04:53:56,405 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf2/c543f3f56d754808a11f335a526021df, entries=10, sequenceid=71, filesize=5.0 K 2023-05-10 04:53:56,406 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 4dcb62249031b560a1bc5d03e43ac855 in 39ms, sequenceid=71, compaction requested=true 2023-05-10 04:53:56,406 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 4dcb62249031b560a1bc5d03e43ac855: 2023-05-10 04:53:56,406 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1683694434423.4dcb62249031b560a1bc5d03e43ac855. 2023-05-10 04:53:56,406 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-10 04:53:56,406 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-10 04:53:56,406 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:56,406 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-05-10 04:53:56,406 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37823,1683694419534' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-05-10 04:53:56,408 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:56,408 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-10 04:53:56,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:56,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:53:56,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:53:56,408 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-10 04:53:56,408 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:53:56,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:53:56,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:53:56,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-10 04:53:56,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:56,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:56,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:56,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:53:56,410 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37823,1683694419534' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-05-10 04:53:56,410 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7d5976b9[Count = 0] remaining members to acquire global barrier 2023-05-10 04:53:56,410 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-05-10 04:53:56,410 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-10 04:53:56,412 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-10 04:53:56,412 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-10 04:53:56,412 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-10 04:53:56,412 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-10 04:53:56,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-10 04:53:56,412 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-10 04:53:56,412 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-05-10 04:53:56,412 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:56,412 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-10 04:53:56,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-10 04:53:56,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-10 04:53:56,412 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-05-10 04:53:56,413 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-05-10 04:53:56,413 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-05-10 04:53:56,412 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-10 04:53:56,412 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-05-10 04:53:56,413 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,37995,1683694419499' in zk 2023-05-10 04:53:56,413 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-05-10 04:53:56,413 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,37823,1683694419534' in zk 2023-05-10 04:53:56,413 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,36665,1683694419452' in zk 2023-05-10 04:53:56,414 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:56,414 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-05-10 04:53:56,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:56,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:53:56,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:53:56,414 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:53:56,414 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-05-10 04:53:56,415 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-05-10 04:53:56,415 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-05-10 04:53:56,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:53:56,415 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:53:56,415 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:53:56,415 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-05-10 04:53:56,415 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-05-10 04:53:56,415 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:53:56,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-10 04:53:56,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:56,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:56,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:56,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:53:56,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-10 04:53:56,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:56,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:56,417 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:56,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase4.apache.org,37823,1683694419534': 2023-05-10 04:53:56,418 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37823,1683694419534' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-05-10 04:53:56,418 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-05-10 04:53:56,418 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-10 04:53:56,418 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-10 04:53:56,418 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-05-10 04:53:56,418 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-10 04:53:56,420 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-10 04:53:56,420 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-10 04:53:56,420 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-10 04:53:56,420 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-10 04:53:56,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-10 04:53:56,420 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-10 04:53:56,420 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:53:56,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-10 04:53:56,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:53:56,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:53:56,420 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-10 04:53:56,420 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:53:56,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-10 04:53:56,420 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:53:56,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:53:56,420 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-10 04:53:56,420 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:53:56,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-10 04:53:56,420 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:56,421 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:53:56,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:53:56,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-10 04:53:56,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:53:56,421 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:53:56,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:53:56,421 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:56,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-10 04:53:56,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-10 04:53:56,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-10 04:53:56,421 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:56,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:53:56,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-10 04:53:56,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:56,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:56,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:56,423 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:56,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:53:56,423 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:56,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-05-10 04:53:56,423 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:56,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:56,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:56,424 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:56,430 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:53:56,430 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:53:56,430 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:53:56,430 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:53:56,431 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-05-10 04:53:56,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:53:56,430 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:53:56,430 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:53:56,430 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:53:56,430 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:56,431 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:53:56,431 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:53:56,431 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-10 04:53:56,431 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:53:56,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:53:56,431 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:53:56,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:53:56,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:53:56,431 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-10 04:53:56,431 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:56,431 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:56,431 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:53:56,431 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-05-10 04:53:56,431 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:53:56,431 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-10 04:53:56,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:53:56,431 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-05-10 04:53:56,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:53:56,432 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:53:56,432 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-10 04:53:56,432 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:53:56,432 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:56,432 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-05-10 04:53:56,432 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-05-10 04:53:56,532 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-10 04:53:56,533 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-10 04:53:56,794 WARN [Listener at localhost/45007] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: true 2023-05-10 04:53:56,795 DEBUG [Listener at localhost/45007] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:56,796 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(1029): Stored pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:53:56,797 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:53:56,797 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:53:56,797 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:53:56,798 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-10 04:53:56,798 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:53:56,798 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-10 04:53:56,799 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-10 04:53:57,046 INFO [Listener at localhost/45007] procedure.MasterProcedureTestingUtility(417): Restart 2 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:53:57,046 INFO [Listener at localhost/45007] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-10 04:53:57,046 INFO [Listener at localhost/45007] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-10 04:53:57,047 INFO [Listener at localhost/45007] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-10 04:53:57,047 ERROR [Listener at localhost/45007] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-7,5,PEWorkerGroup] Thread[HFileArchiver-8,5,PEWorkerGroup] 2023-05-10 04:53:57,047 INFO [Listener at localhost/45007] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-10 04:53:57,047 INFO [Listener at localhost/45007] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-10 04:53:57,047 INFO [Listener at localhost/45007] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-10 04:53:57,047 INFO [Listener at localhost/45007] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-10 04:53:57,048 INFO [Listener at localhost/45007] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-10 04:53:57,049 DEBUG [Listener at localhost/45007] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[] 2023-05-10 04:53:57,049 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-10 04:53:57,049 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-10 04:53:57,050 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-10 04:53:57,050 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-10 04:53:57,050 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:53:57,051 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=93c2cc329b1fe1893cfbc28c56960522, daughterA=19040b6538a6bdeed7c833cfb44614b6, daughterB=054f69c06f126aad97b290410d5d6188 2023-05-10 04:53:57,051 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:53:57,051 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:53:57,051 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-10 04:53:57,051 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6c70954228a0bbe97dfdc76861367c66, daughterA=75e2592cb2ac08bd9f617bb9d0a0c896, daughterB=b90fd344aff14f0728a80c3d09539e54 2023-05-10 04:53:57,051 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-05-10 04:53:57,052 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-10 04:53:57,052 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-10 04:53:57,052 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:53:57,052 DEBUG [Listener at localhost/45007] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 held the lock before restarting, call acquireLock to restore it. 2023-05-10 04:53:57,052 INFO [Listener at localhost/45007] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:53:57,052 INFO [Listener at localhost/45007] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:53:57,052 INFO [Listener at localhost/45007] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:53:57,053 INFO [Listener at localhost/45007] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-05-10 04:53:57,053 DEBUG [Listener at localhost/45007] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-10 04:53:57,054 DEBUG [Listener at localhost/45007] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,37995,1683694419499, table=hbase:meta, region=1588230740 2023-05-10 04:53:57,054 INFO [Listener at localhost/45007] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683694467054 2023-05-10 04:53:57,054 DEBUG [Listener at localhost/45007] assignment.AssignmentManager(1517): Joining cluster... 2023-05-10 04:53:57,055 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:53:57,058 DEBUG [Listener at localhost/45007] assignment.RegionStateStore(147): Load hbase:meta entry region=ccfda81dc2f13ffc1429bef67d349028, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36665,1683694419452, regionLocation=jenkins-hbase4.apache.org,36665,1683694419452, openSeqNum=2 2023-05-10 04:53:57,058 DEBUG [Listener at localhost/45007] assignment.RegionStateStore(147): Load hbase:meta entry region=4dcb62249031b560a1bc5d03e43ac855, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37823,1683694419534, regionLocation=jenkins-hbase4.apache.org,37823,1683694419534, openSeqNum=2 2023-05-10 04:53:57,058 INFO [Listener at localhost/45007] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-10 04:53:57,058 INFO [Listener at localhost/45007] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683694497058 2023-05-10 04:53:57,058 INFO [Listener at localhost/45007] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683694557058 2023-05-10 04:53:57,058 INFO [Listener at localhost/45007] assignment.AssignmentManager(1536): Joined the cluster in 4 msec 2023-05-10 04:53:57,058 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-10 04:53:57,060 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-10 04:53:57,060 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:53:57,060 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-10 04:53:57,060 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-10 04:53:57,308 INFO [Listener at localhost/45007] procedure.MasterProcedureTestingUtility(417): Restart 3 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:53:57,309 INFO [Listener at localhost/45007] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-10 04:53:57,309 INFO [Listener at localhost/45007] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-10 04:53:57,309 INFO [Listener at localhost/45007] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-10 04:53:57,310 ERROR [Listener at localhost/45007] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-10 04:53:57,310 INFO [Listener at localhost/45007] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-10 04:53:57,310 INFO [Listener at localhost/45007] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-10 04:53:57,310 INFO [Listener at localhost/45007] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-10 04:53:57,310 INFO [Listener at localhost/45007] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-10 04:53:57,311 INFO [Listener at localhost/45007] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-10 04:53:57,312 DEBUG [Listener at localhost/45007] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-05-10 04:53:57,312 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-10 04:53:57,313 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-10 04:53:57,313 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-10 04:53:57,313 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-10 04:53:57,314 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:53:57,314 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=93c2cc329b1fe1893cfbc28c56960522, daughterA=19040b6538a6bdeed7c833cfb44614b6, daughterB=054f69c06f126aad97b290410d5d6188 2023-05-10 04:53:57,314 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:53:57,315 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:53:57,315 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-10 04:53:57,315 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6c70954228a0bbe97dfdc76861367c66, daughterA=75e2592cb2ac08bd9f617bb9d0a0c896, daughterB=b90fd344aff14f0728a80c3d09539e54 2023-05-10 04:53:57,316 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-05-10 04:53:57,316 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-10 04:53:57,316 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-10 04:53:57,317 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:53:57,317 DEBUG [Listener at localhost/45007] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 held the lock before restarting, call acquireLock to restore it. 2023-05-10 04:53:57,317 INFO [Listener at localhost/45007] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:53:57,317 INFO [Listener at localhost/45007] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:53:57,317 INFO [Listener at localhost/45007] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:53:57,317 INFO [Listener at localhost/45007] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-05-10 04:53:57,318 DEBUG [Listener at localhost/45007] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-10 04:53:57,319 DEBUG [Listener at localhost/45007] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,37995,1683694419499, table=hbase:meta, region=1588230740 2023-05-10 04:53:57,319 INFO [Listener at localhost/45007] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683694467319 2023-05-10 04:53:57,320 DEBUG [Listener at localhost/45007] assignment.AssignmentManager(1517): Joining cluster... 2023-05-10 04:53:57,320 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:53:57,323 DEBUG [Listener at localhost/45007] assignment.RegionStateStore(147): Load hbase:meta entry region=ccfda81dc2f13ffc1429bef67d349028, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36665,1683694419452, regionLocation=jenkins-hbase4.apache.org,36665,1683694419452, openSeqNum=2 2023-05-10 04:53:57,323 DEBUG [Listener at localhost/45007] assignment.RegionStateStore(147): Load hbase:meta entry region=4dcb62249031b560a1bc5d03e43ac855, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37823,1683694419534, regionLocation=jenkins-hbase4.apache.org,37823,1683694419534, openSeqNum=2 2023-05-10 04:53:57,323 INFO [Listener at localhost/45007] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-10 04:53:57,323 INFO [Listener at localhost/45007] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683694497323 2023-05-10 04:53:57,323 INFO [Listener at localhost/45007] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683694557323 2023-05-10 04:53:57,323 INFO [Listener at localhost/45007] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-05-10 04:53:57,323 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-10 04:53:57,326 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=4dcb62249031b560a1bc5d03e43ac855, UNASSIGN}] 2023-05-10 04:53:57,326 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-10 04:53:57,326 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:53:57,326 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-10 04:53:57,326 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-10 04:53:57,573 INFO [Listener at localhost/45007] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:53:57,574 INFO [Listener at localhost/45007] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-10 04:53:57,574 INFO [Listener at localhost/45007] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-10 04:53:57,574 INFO [Listener at localhost/45007] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-10 04:53:57,574 ERROR [Listener at localhost/45007] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-10 04:53:57,574 INFO [Listener at localhost/45007] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-10 04:53:57,574 INFO [Listener at localhost/45007] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-10 04:53:57,574 INFO [Listener at localhost/45007] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-10 04:53:57,574 INFO [Listener at localhost/45007] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-10 04:53:57,575 INFO [Listener at localhost/45007] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-10 04:53:57,576 DEBUG [Listener at localhost/45007] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-05-10 04:53:57,576 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-10 04:53:57,577 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-10 04:53:57,577 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-10 04:53:57,577 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-10 04:53:57,577 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:53:57,578 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=93c2cc329b1fe1893cfbc28c56960522, daughterA=19040b6538a6bdeed7c833cfb44614b6, daughterB=054f69c06f126aad97b290410d5d6188 2023-05-10 04:53:57,578 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:53:57,578 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:53:57,578 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-10 04:53:57,578 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6c70954228a0bbe97dfdc76861367c66, daughterA=75e2592cb2ac08bd9f617bb9d0a0c896, daughterB=b90fd344aff14f0728a80c3d09539e54 2023-05-10 04:53:57,579 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-05-10 04:53:57,579 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-10 04:53:57,580 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-10 04:53:57,580 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:53:57,580 DEBUG [Listener at localhost/45007] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 held the lock before restarting, call acquireLock to restore it. 2023-05-10 04:53:57,580 INFO [Listener at localhost/45007] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:53:57,580 INFO [Listener at localhost/45007] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:53:57,580 INFO [Listener at localhost/45007] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:53:57,580 INFO [Listener at localhost/45007] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-05-10 04:53:57,581 DEBUG [Listener at localhost/45007] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-10 04:53:57,582 DEBUG [Listener at localhost/45007] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,37995,1683694419499, table=hbase:meta, region=1588230740 2023-05-10 04:53:57,582 INFO [Listener at localhost/45007] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683694467582 2023-05-10 04:53:57,583 DEBUG [Listener at localhost/45007] assignment.AssignmentManager(1517): Joining cluster... 2023-05-10 04:53:57,583 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:53:57,586 DEBUG [Listener at localhost/45007] assignment.RegionStateStore(147): Load hbase:meta entry region=ccfda81dc2f13ffc1429bef67d349028, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36665,1683694419452, regionLocation=jenkins-hbase4.apache.org,36665,1683694419452, openSeqNum=2 2023-05-10 04:53:57,586 DEBUG [Listener at localhost/45007] assignment.RegionStateStore(147): Load hbase:meta entry region=4dcb62249031b560a1bc5d03e43ac855, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,37823,1683694419534, regionLocation=jenkins-hbase4.apache.org,37823,1683694419534, openSeqNum=2 2023-05-10 04:53:57,586 INFO [Listener at localhost/45007] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-10 04:53:57,587 INFO [Listener at localhost/45007] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683694497586 2023-05-10 04:53:57,587 INFO [Listener at localhost/45007] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683694557587 2023-05-10 04:53:57,587 INFO [Listener at localhost/45007] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-05-10 04:53:57,587 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=4dcb62249031b560a1bc5d03e43ac855, UNASSIGN}] 2023-05-10 04:53:57,587 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-10 04:53:57,589 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=4dcb62249031b560a1bc5d03e43ac855, UNASSIGN 2023-05-10 04:53:57,590 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=4dcb62249031b560a1bc5d03e43ac855, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:57,590 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1683694434423.4dcb62249031b560a1bc5d03e43ac855.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1683694437590"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683694437590"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683694437590"}]},"ts":"1683694437590"} 2023-05-10 04:53:57,592 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 4dcb62249031b560a1bc5d03e43ac855, server=jenkins-hbase4.apache.org,37823,1683694419534}] 2023-05-10 04:53:57,592 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-10 04:53:57,592 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=4dcb62249031b560a1bc5d03e43ac855, UNASSIGN 2023-05-10 04:53:57,592 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-10 04:53:57,592 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=4dcb62249031b560a1bc5d03e43ac855, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-10 04:53:57,837 INFO [Listener at localhost/45007] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:53:57,837 INFO [Listener at localhost/45007] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-10 04:53:57,837 INFO [Listener at localhost/45007] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-10 04:53:57,837 INFO [Listener at localhost/45007] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-10 04:53:57,840 ERROR [Listener at localhost/45007] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-10 04:53:57,841 INFO [Listener at localhost/45007] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-10 04:53:57,841 INFO [Listener at localhost/45007] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-10 04:53:57,841 INFO [Listener at localhost/45007] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-10 04:53:57,841 INFO [Listener at localhost/45007] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-10 04:53:57,842 INFO [Listener at localhost/45007] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-10 04:53:57,843 DEBUG [Listener at localhost/45007] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-05-10 04:53:57,843 DEBUG [Listener at localhost/45007] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-05-10 04:53:57,843 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-10 04:53:57,844 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-10 04:53:57,844 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-10 04:53:57,844 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-10 04:53:57,845 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:53:57,845 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=93c2cc329b1fe1893cfbc28c56960522, daughterA=19040b6538a6bdeed7c833cfb44614b6, daughterB=054f69c06f126aad97b290410d5d6188 2023-05-10 04:53:57,846 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:53:57,846 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:53:57,846 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-10 04:53:57,847 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6c70954228a0bbe97dfdc76861367c66, daughterA=75e2592cb2ac08bd9f617bb9d0a0c896, daughterB=b90fd344aff14f0728a80c3d09539e54 2023-05-10 04:53:57,847 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-05-10 04:53:57,847 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-10 04:53:57,848 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-10 04:53:57,849 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(468): Loading pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:53:57,849 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=4dcb62249031b560a1bc5d03e43ac855, UNASSIGN 2023-05-10 04:53:57,849 DEBUG [Listener at localhost/45007] procedure2.Procedure(966): pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 held the lock before restarting, call acquireLock to restore it. 2023-05-10 04:53:57,849 INFO [Listener at localhost/45007] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:53:57,849 INFO [Listener at localhost/45007] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:53:57,849 INFO [Listener at localhost/45007] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:53:57,849 DEBUG [Listener at localhost/45007] procedure2.Procedure(966): pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=4dcb62249031b560a1bc5d03e43ac855, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-05-10 04:53:57,850 INFO [Listener at localhost/45007] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=4dcb62249031b560a1bc5d03e43ac855, UNASSIGN 2023-05-10 04:53:57,850 INFO [Listener at localhost/45007] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-05-10 04:53:57,851 DEBUG [Listener at localhost/45007] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-10 04:53:57,851 DEBUG [Listener at localhost/45007] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,37995,1683694419499, table=hbase:meta, region=1588230740 2023-05-10 04:53:57,851 INFO [Listener at localhost/45007] assignment.AssignmentManager(305): Attach pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=4dcb62249031b560a1bc5d03e43ac855, UNASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=4dcb62249031b560a1bc5d03e43ac855 to restore RIT 2023-05-10 04:53:57,858 INFO [Listener at localhost/45007] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683694467858 2023-05-10 04:53:57,860 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=4dcb62249031b560a1bc5d03e43ac855, UNASSIGN 2023-05-10 04:53:57,860 DEBUG [Listener at localhost/45007] assignment.AssignmentManager(1517): Joining cluster... 2023-05-10 04:53:57,863 DEBUG [Listener at localhost/45007] assignment.RegionStateStore(147): Load hbase:meta entry region=ccfda81dc2f13ffc1429bef67d349028, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36665,1683694419452, regionLocation=jenkins-hbase4.apache.org,36665,1683694419452, openSeqNum=2 2023-05-10 04:53:57,863 DEBUG [Listener at localhost/45007] assignment.RegionStateStore(147): Load hbase:meta entry region=4dcb62249031b560a1bc5d03e43ac855, regionState=CLOSING, lastHost=jenkins-hbase4.apache.org,37823,1683694419534, regionLocation=jenkins-hbase4.apache.org,37823,1683694419534, openSeqNum=2 2023-05-10 04:53:57,863 INFO [Listener at localhost/45007] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-10 04:53:57,864 INFO [Listener at localhost/45007] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683694497864 2023-05-10 04:53:57,864 INFO [Listener at localhost/45007] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683694557864 2023-05-10 04:53:57,864 INFO [Listener at localhost/45007] assignment.AssignmentManager(1536): Joined the cluster in 4 msec 2023-05-10 04:53:57,864 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=4dcb62249031b560a1bc5d03e43ac855, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:57,864 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1683694434423.4dcb62249031b560a1bc5d03e43ac855.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1683694437864"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683694437864"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683694437864"}]},"ts":"1683694437864"} 2023-05-10 04:53:57,866 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 4dcb62249031b560a1bc5d03e43ac855, server=jenkins-hbase4.apache.org,37823,1683694419534}] 2023-05-10 04:53:57,866 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-10 04:53:58,019 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 4dcb62249031b560a1bc5d03e43ac855 2023-05-10 04:53:58,019 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 4dcb62249031b560a1bc5d03e43ac855, disabling compactions & flushes 2023-05-10 04:53:58,019 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testRollbackAndDoubleExecution,,1683694434423.4dcb62249031b560a1bc5d03e43ac855. 2023-05-10 04:53:58,019 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRollbackAndDoubleExecution,,1683694434423.4dcb62249031b560a1bc5d03e43ac855. 2023-05-10 04:53:58,019 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testRollbackAndDoubleExecution,,1683694434423.4dcb62249031b560a1bc5d03e43ac855. after waiting 0 ms 2023-05-10 04:53:58,020 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testRollbackAndDoubleExecution,,1683694434423.4dcb62249031b560a1bc5d03e43ac855. 2023-05-10 04:53:58,020 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 4dcb62249031b560a1bc5d03e43ac855 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-05-10 04:53:58,029 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/.tmp/cf1/6250e2688ebe49cda50b3b8449609ff3 2023-05-10 04:53:58,049 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/.tmp/cf2/81fa4aef7d904ef1a24ea0776a615d6f 2023-05-10 04:53:58,054 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/.tmp/cf1/6250e2688ebe49cda50b3b8449609ff3 as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf1/6250e2688ebe49cda50b3b8449609ff3 2023-05-10 04:53:58,058 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf1/6250e2688ebe49cda50b3b8449609ff3, entries=8, sequenceid=82, filesize=5.0 K 2023-05-10 04:53:58,059 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/.tmp/cf2/81fa4aef7d904ef1a24ea0776a615d6f as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf2/81fa4aef7d904ef1a24ea0776a615d6f 2023-05-10 04:53:58,064 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf2/81fa4aef7d904ef1a24ea0776a615d6f, entries=8, sequenceid=82, filesize=5.0 K 2023-05-10 04:53:58,065 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 4dcb62249031b560a1bc5d03e43ac855 in 45ms, sequenceid=82, compaction requested=true 2023-05-10 04:53:58,070 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-05-10 04:53:58,070 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-10 04:53:58,071 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testRollbackAndDoubleExecution,,1683694434423.4dcb62249031b560a1bc5d03e43ac855. 2023-05-10 04:53:58,071 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 4dcb62249031b560a1bc5d03e43ac855: 2023-05-10 04:53:58,073 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 4dcb62249031b560a1bc5d03e43ac855 2023-05-10 04:53:58,073 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=4dcb62249031b560a1bc5d03e43ac855, regionState=CLOSED 2023-05-10 04:53:58,074 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackAndDoubleExecution,,1683694434423.4dcb62249031b560a1bc5d03e43ac855.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1683694438073"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683694438073"}]},"ts":"1683694438073"} 2023-05-10 04:53:58,075 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-10 04:53:58,075 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=46, ppid=45, state=SUCCESS, locked=true; CloseRegionProcedure 4dcb62249031b560a1bc5d03e43ac855, server=jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:58,075 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-10 04:53:58,075 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=46, ppid=45, state=SUCCESS, locked=true; CloseRegionProcedure 4dcb62249031b560a1bc5d03e43ac855, server=jenkins-hbase4.apache.org,37823,1683694419534 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-10 04:53:58,114 INFO [Listener at localhost/45007] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:53:58,114 INFO [Listener at localhost/45007] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-10 04:53:58,114 INFO [Listener at localhost/45007] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-10 04:53:58,114 INFO [Listener at localhost/45007] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-10 04:53:58,116 ERROR [Listener at localhost/45007] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-10 04:53:58,116 INFO [Listener at localhost/45007] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-10 04:53:58,116 INFO [Listener at localhost/45007] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-10 04:53:58,116 INFO [Listener at localhost/45007] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-10 04:53:58,116 INFO [Listener at localhost/45007] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-10 04:53:58,116 INFO [Listener at localhost/45007] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-10 04:53:58,117 DEBUG [Listener at localhost/45007] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-05-10 04:53:58,117 DEBUG [Listener at localhost/45007] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-05-10 04:53:58,117 DEBUG [Listener at localhost/45007] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4] 2023-05-10 04:53:58,118 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-10 04:53:58,118 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-10 04:53:58,118 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-10 04:53:58,118 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-10 04:53:58,119 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:53:58,119 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=93c2cc329b1fe1893cfbc28c56960522, daughterA=19040b6538a6bdeed7c833cfb44614b6, daughterB=054f69c06f126aad97b290410d5d6188 2023-05-10 04:53:58,119 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:53:58,119 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:53:58,119 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-10 04:53:58,120 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6c70954228a0bbe97dfdc76861367c66, daughterA=75e2592cb2ac08bd9f617bb9d0a0c896, daughterB=b90fd344aff14f0728a80c3d09539e54 2023-05-10 04:53:58,120 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-05-10 04:53:58,120 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-10 04:53:58,120 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-10 04:53:58,121 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(468): Loading pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:53:58,121 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=4dcb62249031b560a1bc5d03e43ac855, UNASSIGN 2023-05-10 04:53:58,121 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 4dcb62249031b560a1bc5d03e43ac855, server=jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:58,121 DEBUG [Listener at localhost/45007] procedure2.Procedure(966): pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 held the lock before restarting, call acquireLock to restore it. 2023-05-10 04:53:58,121 INFO [Listener at localhost/45007] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:53:58,121 INFO [Listener at localhost/45007] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:53:58,121 INFO [Listener at localhost/45007] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:53:58,121 DEBUG [Listener at localhost/45007] procedure2.Procedure(966): pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=4dcb62249031b560a1bc5d03e43ac855, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-05-10 04:53:58,121 INFO [Listener at localhost/45007] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=4dcb62249031b560a1bc5d03e43ac855, UNASSIGN 2023-05-10 04:53:58,121 DEBUG [Listener at localhost/45007] procedure2.Procedure(966): pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 4dcb62249031b560a1bc5d03e43ac855, server=jenkins-hbase4.apache.org,37823,1683694419534 held the lock before restarting, call acquireLock to restore it. 2023-05-10 04:53:58,121 INFO [Listener at localhost/45007] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-05-10 04:53:58,122 DEBUG [Listener at localhost/45007] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-10 04:53:58,123 DEBUG [Listener at localhost/45007] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,37995,1683694419499, table=hbase:meta, region=1588230740 2023-05-10 04:53:58,123 INFO [Listener at localhost/45007] assignment.AssignmentManager(305): Attach pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=4dcb62249031b560a1bc5d03e43ac855, UNASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=4dcb62249031b560a1bc5d03e43ac855 to restore RIT 2023-05-10 04:53:58,123 INFO [Listener at localhost/45007] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683694468123 2023-05-10 04:53:58,124 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 4dcb62249031b560a1bc5d03e43ac855, server=jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:58,124 DEBUG [Listener at localhost/45007] assignment.AssignmentManager(1517): Joining cluster... 2023-05-10 04:53:58,126 DEBUG [Listener at localhost/45007] assignment.RegionStateStore(147): Load hbase:meta entry region=ccfda81dc2f13ffc1429bef67d349028, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36665,1683694419452, regionLocation=jenkins-hbase4.apache.org,36665,1683694419452, openSeqNum=2 2023-05-10 04:53:58,127 DEBUG [Listener at localhost/45007] assignment.RegionStateStore(147): Load hbase:meta entry region=4dcb62249031b560a1bc5d03e43ac855, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,37823,1683694419534, regionLocation=jenkins-hbase4.apache.org,37823,1683694419534, openSeqNum=2 2023-05-10 04:53:58,127 INFO [Listener at localhost/45007] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-10 04:53:58,127 INFO [Listener at localhost/45007] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683694498127 2023-05-10 04:53:58,127 INFO [Listener at localhost/45007] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683694558127 2023-05-10 04:53:58,127 INFO [Listener at localhost/45007] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-05-10 04:53:58,127 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=4dcb62249031b560a1bc5d03e43ac855, regionState=CLOSED, regionLocation=jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:58,128 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1683694434423.4dcb62249031b560a1bc5d03e43ac855.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1683694438127"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683694438127"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683694438127"}]},"ts":"1683694438127"} 2023-05-10 04:53:58,129 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-10 04:53:58,134 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=46, resume processing ppid=45 2023-05-10 04:53:58,134 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 4dcb62249031b560a1bc5d03e43ac855, server=jenkins-hbase4.apache.org,37823,1683694419534 in 263 msec 2023-05-10 04:53:58,135 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-10 04:53:58,135 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=45, ppid=44, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=4dcb62249031b560a1bc5d03e43ac855, UNASSIGN 2023-05-10 04:53:58,135 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-10 04:53:58,135 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=45, ppid=44, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=4dcb62249031b560a1bc5d03e43ac855, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-10 04:53:58,377 INFO [Listener at localhost/45007] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:53:58,377 INFO [Listener at localhost/45007] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-10 04:53:58,378 INFO [Listener at localhost/45007] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-10 04:53:58,378 INFO [Listener at localhost/45007] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-10 04:53:58,378 ERROR [Listener at localhost/45007] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-10 04:53:58,378 INFO [Listener at localhost/45007] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-10 04:53:58,378 INFO [Listener at localhost/45007] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-10 04:53:58,378 INFO [Listener at localhost/45007] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-10 04:53:58,378 INFO [Listener at localhost/45007] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-10 04:53:58,379 INFO [Listener at localhost/45007] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-10 04:53:58,380 DEBUG [Listener at localhost/45007] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-05-10 04:53:58,380 DEBUG [Listener at localhost/45007] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-05-10 04:53:58,380 DEBUG [Listener at localhost/45007] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-05-10 04:53:58,381 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-10 04:53:58,381 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-10 04:53:58,381 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-10 04:53:58,381 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-10 04:53:58,382 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:53:58,382 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=93c2cc329b1fe1893cfbc28c56960522, daughterA=19040b6538a6bdeed7c833cfb44614b6, daughterB=054f69c06f126aad97b290410d5d6188 2023-05-10 04:53:58,382 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:53:58,382 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:53:58,383 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-10 04:53:58,383 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6c70954228a0bbe97dfdc76861367c66, daughterA=75e2592cb2ac08bd9f617bb9d0a0c896, daughterB=b90fd344aff14f0728a80c3d09539e54 2023-05-10 04:53:58,383 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-05-10 04:53:58,383 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-10 04:53:58,383 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-10 04:53:58,384 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(468): Loading pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:53:58,384 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=4dcb62249031b560a1bc5d03e43ac855, UNASSIGN 2023-05-10 04:53:58,384 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 4dcb62249031b560a1bc5d03e43ac855, server=jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:58,384 DEBUG [Listener at localhost/45007] procedure2.Procedure(966): pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 held the lock before restarting, call acquireLock to restore it. 2023-05-10 04:53:58,384 INFO [Listener at localhost/45007] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:53:58,384 INFO [Listener at localhost/45007] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:53:58,385 INFO [Listener at localhost/45007] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:53:58,385 DEBUG [Listener at localhost/45007] procedure2.Procedure(946): pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=4dcb62249031b560a1bc5d03e43ac855, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-05-10 04:53:58,385 DEBUG [Listener at localhost/45007] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 4dcb62249031b560a1bc5d03e43ac855, server=jenkins-hbase4.apache.org,37823,1683694419534 didn't hold the lock before restarting, skip acquiring lock. 2023-05-10 04:53:58,385 INFO [Listener at localhost/45007] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-05-10 04:53:58,385 DEBUG [Listener at localhost/45007] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-10 04:53:58,386 DEBUG [Listener at localhost/45007] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,37995,1683694419499, table=hbase:meta, region=1588230740 2023-05-10 04:53:58,386 INFO [Listener at localhost/45007] assignment.AssignmentManager(305): Attach pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=4dcb62249031b560a1bc5d03e43ac855, UNASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=4dcb62249031b560a1bc5d03e43ac855 to restore RIT 2023-05-10 04:53:58,387 INFO [Listener at localhost/45007] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683694468387 2023-05-10 04:53:58,388 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=4dcb62249031b560a1bc5d03e43ac855, UNASSIGN 2023-05-10 04:53:58,388 DEBUG [Listener at localhost/45007] assignment.AssignmentManager(1517): Joining cluster... 2023-05-10 04:53:58,390 DEBUG [Listener at localhost/45007] assignment.RegionStateStore(147): Load hbase:meta entry region=ccfda81dc2f13ffc1429bef67d349028, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36665,1683694419452, regionLocation=jenkins-hbase4.apache.org,36665,1683694419452, openSeqNum=2 2023-05-10 04:53:58,390 DEBUG [Listener at localhost/45007] assignment.RegionStateStore(147): Load hbase:meta entry region=4dcb62249031b560a1bc5d03e43ac855, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,37823,1683694419534, regionLocation=jenkins-hbase4.apache.org,37823,1683694419534, openSeqNum=2 2023-05-10 04:53:58,390 INFO [Listener at localhost/45007] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-10 04:53:58,391 INFO [Listener at localhost/45007] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683694498390 2023-05-10 04:53:58,391 INFO [Listener at localhost/45007] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683694558391 2023-05-10 04:53:58,391 INFO [Listener at localhost/45007] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-05-10 04:53:58,391 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=4dcb62249031b560a1bc5d03e43ac855, UNASSIGN 2023-05-10 04:53:58,392 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-10 04:53:58,395 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=45, resume processing ppid=44 2023-05-10 04:53:58,395 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=4dcb62249031b560a1bc5d03e43ac855, UNASSIGN in 805 msec 2023-05-10 04:53:58,398 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-10 04:53:58,398 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:53:58,399 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-10 04:53:58,399 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-10 04:53:58,641 INFO [Listener at localhost/45007] procedure.MasterProcedureTestingUtility(417): Restart 4 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:53:58,641 INFO [Listener at localhost/45007] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-10 04:53:58,641 INFO [Listener at localhost/45007] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-10 04:53:58,641 INFO [Listener at localhost/45007] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-10 04:53:58,641 ERROR [Listener at localhost/45007] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-10 04:53:58,642 INFO [Listener at localhost/45007] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-10 04:53:58,642 INFO [Listener at localhost/45007] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-10 04:53:58,642 INFO [Listener at localhost/45007] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-10 04:53:58,642 INFO [Listener at localhost/45007] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-10 04:53:58,642 INFO [Listener at localhost/45007] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-10 04:53:58,643 DEBUG [Listener at localhost/45007] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-05-10 04:53:58,643 DEBUG [Listener at localhost/45007] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-05-10 04:53:58,644 DEBUG [Listener at localhost/45007] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-05-10 04:53:58,644 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-10 04:53:58,644 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-10 04:53:58,644 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-10 04:53:58,644 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-10 04:53:58,645 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:53:58,645 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=93c2cc329b1fe1893cfbc28c56960522, daughterA=19040b6538a6bdeed7c833cfb44614b6, daughterB=054f69c06f126aad97b290410d5d6188 2023-05-10 04:53:58,645 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:53:58,645 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:53:58,646 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-10 04:53:58,646 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6c70954228a0bbe97dfdc76861367c66, daughterA=75e2592cb2ac08bd9f617bb9d0a0c896, daughterB=b90fd344aff14f0728a80c3d09539e54 2023-05-10 04:53:58,646 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-05-10 04:53:58,646 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-10 04:53:58,646 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-10 04:53:58,647 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:53:58,647 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=4dcb62249031b560a1bc5d03e43ac855, UNASSIGN 2023-05-10 04:53:58,647 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 4dcb62249031b560a1bc5d03e43ac855, server=jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:58,647 DEBUG [Listener at localhost/45007] procedure2.Procedure(946): pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 didn't hold the lock before restarting, skip acquiring lock. 2023-05-10 04:53:58,647 DEBUG [Listener at localhost/45007] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=4dcb62249031b560a1bc5d03e43ac855, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-05-10 04:53:58,647 DEBUG [Listener at localhost/45007] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 4dcb62249031b560a1bc5d03e43ac855, server=jenkins-hbase4.apache.org,37823,1683694419534 didn't hold the lock before restarting, skip acquiring lock. 2023-05-10 04:53:58,647 INFO [Listener at localhost/45007] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-05-10 04:53:58,648 DEBUG [Listener at localhost/45007] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-10 04:53:58,648 DEBUG [Listener at localhost/45007] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,37995,1683694419499, table=hbase:meta, region=1588230740 2023-05-10 04:53:58,649 INFO [Listener at localhost/45007] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683694468649 2023-05-10 04:53:58,651 DEBUG [Listener at localhost/45007] assignment.AssignmentManager(1517): Joining cluster... 2023-05-10 04:53:58,651 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:53:58,653 DEBUG [Listener at localhost/45007] assignment.RegionStateStore(147): Load hbase:meta entry region=ccfda81dc2f13ffc1429bef67d349028, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36665,1683694419452, regionLocation=jenkins-hbase4.apache.org,36665,1683694419452, openSeqNum=2 2023-05-10 04:53:58,653 DEBUG [Listener at localhost/45007] assignment.RegionStateStore(147): Load hbase:meta entry region=4dcb62249031b560a1bc5d03e43ac855, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,37823,1683694419534, regionLocation=jenkins-hbase4.apache.org,37823,1683694419534, openSeqNum=2 2023-05-10 04:53:58,653 INFO [Listener at localhost/45007] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-10 04:53:58,653 INFO [Listener at localhost/45007] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683694498653 2023-05-10 04:53:58,653 INFO [Listener at localhost/45007] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683694558653 2023-05-10 04:53:58,653 INFO [Listener at localhost/45007] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-05-10 04:53:58,654 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:53:58,654 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:53:58,654 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:53:58,659 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-10 04:53:58,668 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=44 splitting 14 storefiles, region=4dcb62249031b560a1bc5d03e43ac855, threads=14 2023-05-10 04:53:58,669 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf2/11c6828a4683438fba9bf4cd76081c5d for region: 4dcb62249031b560a1bc5d03e43ac855 2023-05-10 04:53:58,670 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf2/3d1fe957545445a4befe08b6b30a80dc for region: 4dcb62249031b560a1bc5d03e43ac855 2023-05-10 04:53:58,670 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf2/50823f0b97364b55999bb34d2161b4b9 for region: 4dcb62249031b560a1bc5d03e43ac855 2023-05-10 04:53:58,670 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf2/81fa4aef7d904ef1a24ea0776a615d6f for region: 4dcb62249031b560a1bc5d03e43ac855 2023-05-10 04:53:58,670 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf2/c543f3f56d754808a11f335a526021df for region: 4dcb62249031b560a1bc5d03e43ac855 2023-05-10 04:53:58,670 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf2/d22e75e4a5124b3299106ffa2fc21709 for region: 4dcb62249031b560a1bc5d03e43ac855 2023-05-10 04:53:58,670 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf2/d89cd6883f824b51afd167aaa88a54be for region: 4dcb62249031b560a1bc5d03e43ac855 2023-05-10 04:53:58,670 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf1/118eb6964a944d8bb1efaa8c1933daa3 for region: 4dcb62249031b560a1bc5d03e43ac855 2023-05-10 04:53:58,670 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf1/439486fbe6944c55ac7e2e681288c043 for region: 4dcb62249031b560a1bc5d03e43ac855 2023-05-10 04:53:58,671 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf1/8df5350bb6b84fedabe8af09648762f1 for region: 4dcb62249031b560a1bc5d03e43ac855 2023-05-10 04:53:58,671 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf1/6250e2688ebe49cda50b3b8449609ff3 for region: 4dcb62249031b560a1bc5d03e43ac855 2023-05-10 04:53:58,671 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf1/b348d048ffd647f0b322ec445c520c0e for region: 4dcb62249031b560a1bc5d03e43ac855 2023-05-10 04:53:58,671 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf1/be588b1c116843cb87e38baf82c3a2db for region: 4dcb62249031b560a1bc5d03e43ac855 2023-05-10 04:53:58,673 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf1/fc0d292a16e34289b813715b5a9e56bf for region: 4dcb62249031b560a1bc5d03e43ac855 2023-05-10 04:53:58,866 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf2/50823f0b97364b55999bb34d2161b4b9 for region: 4dcb62249031b560a1bc5d03e43ac855 2023-05-10 04:53:58,878 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf1/118eb6964a944d8bb1efaa8c1933daa3 for region: 4dcb62249031b560a1bc5d03e43ac855 2023-05-10 04:53:58,879 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf2/11c6828a4683438fba9bf4cd76081c5d for region: 4dcb62249031b560a1bc5d03e43ac855 2023-05-10 04:53:58,891 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf1/6250e2688ebe49cda50b3b8449609ff3 for region: 4dcb62249031b560a1bc5d03e43ac855 2023-05-10 04:53:58,898 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-05-10 04:53:58,901 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf2/81fa4aef7d904ef1a24ea0776a615d6f for region: 4dcb62249031b560a1bc5d03e43ac855 2023-05-10 04:53:58,902 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf2/3d1fe957545445a4befe08b6b30a80dc for region: 4dcb62249031b560a1bc5d03e43ac855 2023-05-10 04:53:58,919 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf2/d22e75e4a5124b3299106ffa2fc21709 for region: 4dcb62249031b560a1bc5d03e43ac855 2023-05-10 04:53:58,919 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf1/fc0d292a16e34289b813715b5a9e56bf for region: 4dcb62249031b560a1bc5d03e43ac855 2023-05-10 04:53:58,944 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf1/b348d048ffd647f0b322ec445c520c0e for region: 4dcb62249031b560a1bc5d03e43ac855 2023-05-10 04:53:58,946 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf2/c543f3f56d754808a11f335a526021df for region: 4dcb62249031b560a1bc5d03e43ac855 2023-05-10 04:53:58,947 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf2/d89cd6883f824b51afd167aaa88a54be for region: 4dcb62249031b560a1bc5d03e43ac855 2023-05-10 04:53:58,956 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf1/be588b1c116843cb87e38baf82c3a2db for region: 4dcb62249031b560a1bc5d03e43ac855 2023-05-10 04:53:58,957 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf1/439486fbe6944c55ac7e2e681288c043 for region: 4dcb62249031b560a1bc5d03e43ac855 2023-05-10 04:53:58,961 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf1/8df5350bb6b84fedabe8af09648762f1 for region: 4dcb62249031b560a1bc5d03e43ac855 2023-05-10 04:53:58,961 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=44 split storefiles for region 4dcb62249031b560a1bc5d03e43ac855 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-05-10 04:53:59,011 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-10 04:53:59,011 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:53:59,011 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-10 04:53:59,011 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-10 04:53:59,154 INFO [Listener at localhost/45007] procedure.MasterProcedureTestingUtility(417): Restart 5 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:53:59,154 INFO [Listener at localhost/45007] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-10 04:53:59,154 INFO [Listener at localhost/45007] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-10 04:53:59,154 INFO [Listener at localhost/45007] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-10 04:53:59,155 ERROR [Listener at localhost/45007] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-10 04:53:59,155 INFO [Listener at localhost/45007] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-10 04:53:59,155 INFO [Listener at localhost/45007] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-10 04:53:59,155 INFO [Listener at localhost/45007] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-10 04:53:59,155 INFO [Listener at localhost/45007] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-10 04:53:59,156 INFO [Listener at localhost/45007] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-10 04:53:59,157 DEBUG [Listener at localhost/45007] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-05-10 04:53:59,157 DEBUG [Listener at localhost/45007] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-05-10 04:53:59,157 DEBUG [Listener at localhost/45007] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-05-10 04:53:59,158 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-10 04:53:59,158 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-10 04:53:59,158 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-10 04:53:59,158 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-10 04:53:59,159 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:53:59,159 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=93c2cc329b1fe1893cfbc28c56960522, daughterA=19040b6538a6bdeed7c833cfb44614b6, daughterB=054f69c06f126aad97b290410d5d6188 2023-05-10 04:53:59,159 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:53:59,159 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:53:59,159 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-10 04:53:59,160 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6c70954228a0bbe97dfdc76861367c66, daughterA=75e2592cb2ac08bd9f617bb9d0a0c896, daughterB=b90fd344aff14f0728a80c3d09539e54 2023-05-10 04:53:59,160 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-05-10 04:53:59,160 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-10 04:53:59,160 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-10 04:53:59,161 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:53:59,161 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=4dcb62249031b560a1bc5d03e43ac855, UNASSIGN 2023-05-10 04:53:59,161 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 4dcb62249031b560a1bc5d03e43ac855, server=jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:59,161 DEBUG [Listener at localhost/45007] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 held the lock before restarting, call acquireLock to restore it. 2023-05-10 04:53:59,161 INFO [Listener at localhost/45007] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:53:59,161 INFO [Listener at localhost/45007] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:53:59,161 INFO [Listener at localhost/45007] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:53:59,161 DEBUG [Listener at localhost/45007] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=4dcb62249031b560a1bc5d03e43ac855, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-05-10 04:53:59,161 DEBUG [Listener at localhost/45007] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 4dcb62249031b560a1bc5d03e43ac855, server=jenkins-hbase4.apache.org,37823,1683694419534 didn't hold the lock before restarting, skip acquiring lock. 2023-05-10 04:53:59,161 INFO [Listener at localhost/45007] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-05-10 04:53:59,162 DEBUG [Listener at localhost/45007] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-10 04:53:59,163 DEBUG [Listener at localhost/45007] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,37995,1683694419499, table=hbase:meta, region=1588230740 2023-05-10 04:53:59,163 INFO [Listener at localhost/45007] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683694469163 2023-05-10 04:53:59,164 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:53:59,164 DEBUG [Listener at localhost/45007] assignment.AssignmentManager(1517): Joining cluster... 2023-05-10 04:53:59,167 DEBUG [Listener at localhost/45007] assignment.RegionStateStore(147): Load hbase:meta entry region=ccfda81dc2f13ffc1429bef67d349028, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36665,1683694419452, regionLocation=jenkins-hbase4.apache.org,36665,1683694419452, openSeqNum=2 2023-05-10 04:53:59,167 DEBUG [Listener at localhost/45007] assignment.RegionStateStore(147): Load hbase:meta entry region=4dcb62249031b560a1bc5d03e43ac855, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,37823,1683694419534, regionLocation=jenkins-hbase4.apache.org,37823,1683694419534, openSeqNum=2 2023-05-10 04:53:59,167 INFO [Listener at localhost/45007] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-10 04:53:59,167 INFO [Listener at localhost/45007] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683694499167 2023-05-10 04:53:59,167 INFO [Listener at localhost/45007] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683694559167 2023-05-10 04:53:59,167 INFO [Listener at localhost/45007] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-05-10 04:53:59,176 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=44 splitting 14 storefiles, region=4dcb62249031b560a1bc5d03e43ac855, threads=14 2023-05-10 04:53:59,177 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf2/11c6828a4683438fba9bf4cd76081c5d for region: 4dcb62249031b560a1bc5d03e43ac855 2023-05-10 04:53:59,177 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf2/3d1fe957545445a4befe08b6b30a80dc for region: 4dcb62249031b560a1bc5d03e43ac855 2023-05-10 04:53:59,177 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf2/50823f0b97364b55999bb34d2161b4b9 for region: 4dcb62249031b560a1bc5d03e43ac855 2023-05-10 04:53:59,178 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf2/81fa4aef7d904ef1a24ea0776a615d6f for region: 4dcb62249031b560a1bc5d03e43ac855 2023-05-10 04:53:59,178 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf2/c543f3f56d754808a11f335a526021df for region: 4dcb62249031b560a1bc5d03e43ac855 2023-05-10 04:53:59,179 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf2/d22e75e4a5124b3299106ffa2fc21709 for region: 4dcb62249031b560a1bc5d03e43ac855 2023-05-10 04:53:59,179 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf2/d89cd6883f824b51afd167aaa88a54be for region: 4dcb62249031b560a1bc5d03e43ac855 2023-05-10 04:53:59,179 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf1/118eb6964a944d8bb1efaa8c1933daa3 for region: 4dcb62249031b560a1bc5d03e43ac855 2023-05-10 04:53:59,179 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf1/439486fbe6944c55ac7e2e681288c043 for region: 4dcb62249031b560a1bc5d03e43ac855 2023-05-10 04:53:59,184 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf1/6250e2688ebe49cda50b3b8449609ff3 for region: 4dcb62249031b560a1bc5d03e43ac855 2023-05-10 04:53:59,184 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf1/b348d048ffd647f0b322ec445c520c0e for region: 4dcb62249031b560a1bc5d03e43ac855 2023-05-10 04:53:59,184 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf1/8df5350bb6b84fedabe8af09648762f1 for region: 4dcb62249031b560a1bc5d03e43ac855 2023-05-10 04:53:59,184 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf1/be588b1c116843cb87e38baf82c3a2db for region: 4dcb62249031b560a1bc5d03e43ac855 2023-05-10 04:53:59,184 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf1/fc0d292a16e34289b813715b5a9e56bf for region: 4dcb62249031b560a1bc5d03e43ac855 2023-05-10 04:53:59,362 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf1/439486fbe6944c55ac7e2e681288c043 for region: 4dcb62249031b560a1bc5d03e43ac855 2023-05-10 04:53:59,363 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf2/50823f0b97364b55999bb34d2161b4b9 for region: 4dcb62249031b560a1bc5d03e43ac855 2023-05-10 04:53:59,364 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf1/118eb6964a944d8bb1efaa8c1933daa3 for region: 4dcb62249031b560a1bc5d03e43ac855 2023-05-10 04:53:59,375 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf1/6250e2688ebe49cda50b3b8449609ff3 for region: 4dcb62249031b560a1bc5d03e43ac855 2023-05-10 04:53:59,378 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf1/b348d048ffd647f0b322ec445c520c0e for region: 4dcb62249031b560a1bc5d03e43ac855 2023-05-10 04:53:59,378 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf2/11c6828a4683438fba9bf4cd76081c5d for region: 4dcb62249031b560a1bc5d03e43ac855 2023-05-10 04:53:59,383 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf2/c543f3f56d754808a11f335a526021df for region: 4dcb62249031b560a1bc5d03e43ac855 2023-05-10 04:53:59,398 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf1/be588b1c116843cb87e38baf82c3a2db for region: 4dcb62249031b560a1bc5d03e43ac855 2023-05-10 04:53:59,415 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf1/8df5350bb6b84fedabe8af09648762f1 for region: 4dcb62249031b560a1bc5d03e43ac855 2023-05-10 04:53:59,415 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf2/3d1fe957545445a4befe08b6b30a80dc for region: 4dcb62249031b560a1bc5d03e43ac855 2023-05-10 04:53:59,422 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf1/fc0d292a16e34289b813715b5a9e56bf for region: 4dcb62249031b560a1bc5d03e43ac855 2023-05-10 04:53:59,426 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf2/d89cd6883f824b51afd167aaa88a54be for region: 4dcb62249031b560a1bc5d03e43ac855 2023-05-10 04:53:59,426 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf2/81fa4aef7d904ef1a24ea0776a615d6f for region: 4dcb62249031b560a1bc5d03e43ac855 2023-05-10 04:53:59,429 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf2/d22e75e4a5124b3299106ffa2fc21709 for region: 4dcb62249031b560a1bc5d03e43ac855 2023-05-10 04:53:59,429 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=44 split storefiles for region 4dcb62249031b560a1bc5d03e43ac855 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-05-10 04:53:59,474 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-10 04:53:59,479 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/0bdfce2d16e0db079723da4113f9942f/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-05-10 04:53:59,481 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/163c5ab21f256d68dd3647b5b3519220/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-05-10 04:53:59,481 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-10 04:53:59,481 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:53:59,481 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-10 04:53:59,481 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-10 04:53:59,668 INFO [Listener at localhost/45007] procedure.MasterProcedureTestingUtility(417): Restart 6 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:53:59,668 INFO [Listener at localhost/45007] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-10 04:53:59,668 INFO [Listener at localhost/45007] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-10 04:53:59,668 INFO [Listener at localhost/45007] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-10 04:53:59,669 ERROR [Listener at localhost/45007] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-10 04:53:59,669 INFO [Listener at localhost/45007] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-10 04:53:59,669 INFO [Listener at localhost/45007] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-10 04:53:59,669 INFO [Listener at localhost/45007] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-10 04:53:59,669 INFO [Listener at localhost/45007] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-10 04:53:59,670 INFO [Listener at localhost/45007] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-10 04:53:59,671 DEBUG [Listener at localhost/45007] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-05-10 04:53:59,671 DEBUG [Listener at localhost/45007] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-05-10 04:53:59,671 DEBUG [Listener at localhost/45007] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-05-10 04:53:59,672 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-10 04:53:59,672 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-10 04:53:59,672 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-10 04:53:59,672 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-10 04:53:59,673 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:53:59,673 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=93c2cc329b1fe1893cfbc28c56960522, daughterA=19040b6538a6bdeed7c833cfb44614b6, daughterB=054f69c06f126aad97b290410d5d6188 2023-05-10 04:53:59,673 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:53:59,673 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:53:59,673 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-10 04:53:59,674 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6c70954228a0bbe97dfdc76861367c66, daughterA=75e2592cb2ac08bd9f617bb9d0a0c896, daughterB=b90fd344aff14f0728a80c3d09539e54 2023-05-10 04:53:59,674 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-05-10 04:53:59,674 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-10 04:53:59,674 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-10 04:53:59,674 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:53:59,675 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=4dcb62249031b560a1bc5d03e43ac855, UNASSIGN 2023-05-10 04:53:59,675 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 4dcb62249031b560a1bc5d03e43ac855, server=jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:59,678 DEBUG [Listener at localhost/45007] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 held the lock before restarting, call acquireLock to restore it. 2023-05-10 04:53:59,679 INFO [Listener at localhost/45007] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:53:59,679 INFO [Listener at localhost/45007] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:53:59,679 INFO [Listener at localhost/45007] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:53:59,679 DEBUG [Listener at localhost/45007] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=4dcb62249031b560a1bc5d03e43ac855, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-05-10 04:53:59,679 DEBUG [Listener at localhost/45007] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 4dcb62249031b560a1bc5d03e43ac855, server=jenkins-hbase4.apache.org,37823,1683694419534 didn't hold the lock before restarting, skip acquiring lock. 2023-05-10 04:53:59,679 INFO [Listener at localhost/45007] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 8 msec 2023-05-10 04:53:59,680 DEBUG [Listener at localhost/45007] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-10 04:53:59,681 DEBUG [Listener at localhost/45007] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,37995,1683694419499, table=hbase:meta, region=1588230740 2023-05-10 04:53:59,687 INFO [Listener at localhost/45007] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683694469687 2023-05-10 04:53:59,688 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:53:59,688 DEBUG [Listener at localhost/45007] assignment.AssignmentManager(1517): Joining cluster... 2023-05-10 04:53:59,691 DEBUG [Listener at localhost/45007] assignment.RegionStateStore(147): Load hbase:meta entry region=ccfda81dc2f13ffc1429bef67d349028, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36665,1683694419452, regionLocation=jenkins-hbase4.apache.org,36665,1683694419452, openSeqNum=2 2023-05-10 04:53:59,692 DEBUG [Listener at localhost/45007] assignment.RegionStateStore(147): Load hbase:meta entry region=4dcb62249031b560a1bc5d03e43ac855, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,37823,1683694419534, regionLocation=jenkins-hbase4.apache.org,37823,1683694419534, openSeqNum=2 2023-05-10 04:53:59,692 INFO [Listener at localhost/45007] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-10 04:53:59,692 INFO [Listener at localhost/45007] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683694499692 2023-05-10 04:53:59,692 INFO [Listener at localhost/45007] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683694559692 2023-05-10 04:53:59,692 INFO [Listener at localhost/45007] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-05-10 04:53:59,695 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-10 04:53:59,699 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-10 04:53:59,699 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:53:59,699 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-10 04:53:59,699 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-10 04:53:59,943 INFO [Listener at localhost/45007] procedure.MasterProcedureTestingUtility(509): Restart 0 rollback state: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:53:59,943 INFO [Listener at localhost/45007] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-10 04:53:59,943 INFO [Listener at localhost/45007] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-10 04:53:59,943 INFO [Listener at localhost/45007] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-10 04:53:59,944 ERROR [Listener at localhost/45007] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-10 04:53:59,944 INFO [Listener at localhost/45007] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-10 04:53:59,944 INFO [Listener at localhost/45007] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-10 04:53:59,944 INFO [Listener at localhost/45007] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-10 04:53:59,945 INFO [Listener at localhost/45007] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-10 04:53:59,945 INFO [Listener at localhost/45007] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-10 04:53:59,946 DEBUG [Listener at localhost/45007] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-05-10 04:53:59,946 DEBUG [Listener at localhost/45007] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-05-10 04:53:59,946 DEBUG [Listener at localhost/45007] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-05-10 04:53:59,947 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-10 04:53:59,947 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-10 04:53:59,947 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-10 04:53:59,947 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-10 04:53:59,948 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:53:59,948 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=93c2cc329b1fe1893cfbc28c56960522, daughterA=19040b6538a6bdeed7c833cfb44614b6, daughterB=054f69c06f126aad97b290410d5d6188 2023-05-10 04:53:59,949 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:53:59,949 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:53:59,949 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-10 04:53:59,950 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6c70954228a0bbe97dfdc76861367c66, daughterA=75e2592cb2ac08bd9f617bb9d0a0c896, daughterB=b90fd344aff14f0728a80c3d09539e54 2023-05-10 04:53:59,950 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-05-10 04:53:59,950 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-10 04:53:59,950 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-10 04:53:59,951 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:53:59,951 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=4dcb62249031b560a1bc5d03e43ac855, UNASSIGN 2023-05-10 04:53:59,951 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 4dcb62249031b560a1bc5d03e43ac855, server=jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:53:59,951 DEBUG [Listener at localhost/45007] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 held the lock before restarting, call acquireLock to restore it. 2023-05-10 04:53:59,951 INFO [Listener at localhost/45007] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:53:59,951 INFO [Listener at localhost/45007] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:53:59,951 INFO [Listener at localhost/45007] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:53:59,951 DEBUG [Listener at localhost/45007] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=4dcb62249031b560a1bc5d03e43ac855, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-05-10 04:53:59,951 DEBUG [Listener at localhost/45007] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 4dcb62249031b560a1bc5d03e43ac855, server=jenkins-hbase4.apache.org,37823,1683694419534 didn't hold the lock before restarting, skip acquiring lock. 2023-05-10 04:53:59,951 DEBUG [Listener at localhost/45007] procedure2.StateMachineProcedure(225): Abort requested for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:53:59,951 INFO [Listener at localhost/45007] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-05-10 04:53:59,952 DEBUG [Listener at localhost/45007] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-10 04:53:59,952 DEBUG [Listener at localhost/45007] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,37995,1683694419499, table=hbase:meta, region=1588230740 2023-05-10 04:53:59,953 INFO [Listener at localhost/45007] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683694469953 2023-05-10 04:53:59,954 DEBUG [Listener at localhost/45007] assignment.AssignmentManager(1517): Joining cluster... 2023-05-10 04:53:59,955 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:53:59,957 DEBUG [Listener at localhost/45007] assignment.RegionStateStore(147): Load hbase:meta entry region=ccfda81dc2f13ffc1429bef67d349028, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36665,1683694419452, regionLocation=jenkins-hbase4.apache.org,36665,1683694419452, openSeqNum=2 2023-05-10 04:53:59,957 DEBUG [Listener at localhost/45007] assignment.RegionStateStore(147): Load hbase:meta entry region=4dcb62249031b560a1bc5d03e43ac855, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,37823,1683694419534, regionLocation=jenkins-hbase4.apache.org,37823,1683694419534, openSeqNum=2 2023-05-10 04:53:59,957 INFO [Listener at localhost/45007] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-10 04:53:59,957 INFO [Listener at localhost/45007] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683694499957 2023-05-10 04:53:59,958 INFO [Listener at localhost/45007] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683694559958 2023-05-10 04:53:59,958 INFO [Listener at localhost/45007] assignment.AssignmentManager(1536): Joined the cluster in 4 msec 2023-05-10 04:53:59,958 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-10 04:53:59,970 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-10 04:53:59,970 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-05-10 04:53:59,970 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-10 04:53:59,971 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-10 04:54:00,209 INFO [Listener at localhost/45007] procedure.MasterProcedureTestingUtility(509): Restart 1 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:54:00,209 INFO [Listener at localhost/45007] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-10 04:54:00,209 INFO [Listener at localhost/45007] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-10 04:54:00,209 INFO [Listener at localhost/45007] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-10 04:54:00,210 ERROR [Listener at localhost/45007] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-10 04:54:00,210 INFO [Listener at localhost/45007] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-10 04:54:00,210 INFO [Listener at localhost/45007] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-10 04:54:00,210 INFO [Listener at localhost/45007] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-10 04:54:00,210 INFO [Listener at localhost/45007] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-10 04:54:00,211 INFO [Listener at localhost/45007] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-10 04:54:00,212 DEBUG [Listener at localhost/45007] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10] 2023-05-10 04:54:00,212 DEBUG [Listener at localhost/45007] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-05-10 04:54:00,212 DEBUG [Listener at localhost/45007] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-05-10 04:54:00,212 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-10 04:54:00,213 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-10 04:54:00,213 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-10 04:54:00,213 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-10 04:54:00,213 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:54:00,214 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=93c2cc329b1fe1893cfbc28c56960522, daughterA=19040b6538a6bdeed7c833cfb44614b6, daughterB=054f69c06f126aad97b290410d5d6188 2023-05-10 04:54:00,214 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:54:00,214 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:54:00,214 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-10 04:54:00,214 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6c70954228a0bbe97dfdc76861367c66, daughterA=75e2592cb2ac08bd9f617bb9d0a0c896, daughterB=b90fd344aff14f0728a80c3d09539e54 2023-05-10 04:54:00,214 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-05-10 04:54:00,214 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-10 04:54:00,215 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-10 04:54:00,215 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:54:00,216 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=4dcb62249031b560a1bc5d03e43ac855, UNASSIGN 2023-05-10 04:54:00,216 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 4dcb62249031b560a1bc5d03e43ac855, server=jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:54:00,216 DEBUG [Listener at localhost/45007] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 held the lock before restarting, call acquireLock to restore it. 2023-05-10 04:54:00,216 INFO [Listener at localhost/45007] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:54:00,216 INFO [Listener at localhost/45007] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:54:00,216 INFO [Listener at localhost/45007] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:54:00,216 DEBUG [Listener at localhost/45007] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=4dcb62249031b560a1bc5d03e43ac855, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-05-10 04:54:00,216 DEBUG [Listener at localhost/45007] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 4dcb62249031b560a1bc5d03e43ac855, server=jenkins-hbase4.apache.org,37823,1683694419534 didn't hold the lock before restarting, skip acquiring lock. 2023-05-10 04:54:00,216 INFO [Listener at localhost/45007] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-05-10 04:54:00,217 DEBUG [Listener at localhost/45007] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-10 04:54:00,217 DEBUG [Listener at localhost/45007] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,37995,1683694419499, table=hbase:meta, region=1588230740 2023-05-10 04:54:00,218 INFO [Listener at localhost/45007] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683694470217 2023-05-10 04:54:00,220 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:54:00,221 DEBUG [Listener at localhost/45007] assignment.AssignmentManager(1517): Joining cluster... 2023-05-10 04:54:00,223 DEBUG [Listener at localhost/45007] assignment.RegionStateStore(147): Load hbase:meta entry region=ccfda81dc2f13ffc1429bef67d349028, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36665,1683694419452, regionLocation=jenkins-hbase4.apache.org,36665,1683694419452, openSeqNum=2 2023-05-10 04:54:00,223 DEBUG [Listener at localhost/45007] assignment.RegionStateStore(147): Load hbase:meta entry region=4dcb62249031b560a1bc5d03e43ac855, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,37823,1683694419534, regionLocation=jenkins-hbase4.apache.org,37823,1683694419534, openSeqNum=2 2023-05-10 04:54:00,224 INFO [Listener at localhost/45007] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-10 04:54:00,224 INFO [Listener at localhost/45007] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683694500224 2023-05-10 04:54:00,224 INFO [Listener at localhost/45007] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683694560224 2023-05-10 04:54:00,224 INFO [Listener at localhost/45007] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-05-10 04:54:00,224 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-10 04:54:00,226 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-10 04:54:00,226 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-05-10 04:54:00,226 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-10 04:54:00,227 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-10 04:54:00,475 INFO [Listener at localhost/45007] procedure.MasterProcedureTestingUtility(509): Restart 2 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:54:00,475 INFO [Listener at localhost/45007] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-10 04:54:00,475 INFO [Listener at localhost/45007] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-10 04:54:00,475 INFO [Listener at localhost/45007] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-10 04:54:00,475 ERROR [Listener at localhost/45007] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-10 04:54:00,476 INFO [Listener at localhost/45007] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-10 04:54:00,476 INFO [Listener at localhost/45007] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-10 04:54:00,476 INFO [Listener at localhost/45007] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-10 04:54:00,476 INFO [Listener at localhost/45007] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-10 04:54:00,476 INFO [Listener at localhost/45007] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-10 04:54:00,477 DEBUG [Listener at localhost/45007] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-05-10 04:54:00,477 DEBUG [Listener at localhost/45007] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-05-10 04:54:00,478 DEBUG [Listener at localhost/45007] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-05-10 04:54:00,478 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-10 04:54:00,478 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-10 04:54:00,479 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-10 04:54:00,479 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-10 04:54:00,479 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:54:00,480 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=93c2cc329b1fe1893cfbc28c56960522, daughterA=19040b6538a6bdeed7c833cfb44614b6, daughterB=054f69c06f126aad97b290410d5d6188 2023-05-10 04:54:00,480 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:54:00,480 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:54:00,480 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-10 04:54:00,481 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6c70954228a0bbe97dfdc76861367c66, daughterA=75e2592cb2ac08bd9f617bb9d0a0c896, daughterB=b90fd344aff14f0728a80c3d09539e54 2023-05-10 04:54:00,481 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-05-10 04:54:00,481 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-10 04:54:00,481 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-10 04:54:00,482 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:54:00,482 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=4dcb62249031b560a1bc5d03e43ac855, UNASSIGN 2023-05-10 04:54:00,482 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 4dcb62249031b560a1bc5d03e43ac855, server=jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:54:00,482 DEBUG [Listener at localhost/45007] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 held the lock before restarting, call acquireLock to restore it. 2023-05-10 04:54:00,482 INFO [Listener at localhost/45007] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:54:00,482 INFO [Listener at localhost/45007] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:54:00,483 INFO [Listener at localhost/45007] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:54:00,483 DEBUG [Listener at localhost/45007] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=4dcb62249031b560a1bc5d03e43ac855, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-05-10 04:54:00,483 DEBUG [Listener at localhost/45007] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 4dcb62249031b560a1bc5d03e43ac855, server=jenkins-hbase4.apache.org,37823,1683694419534 didn't hold the lock before restarting, skip acquiring lock. 2023-05-10 04:54:00,483 INFO [Listener at localhost/45007] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-05-10 04:54:00,483 DEBUG [Listener at localhost/45007] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-10 04:54:00,484 DEBUG [Listener at localhost/45007] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,37995,1683694419499, table=hbase:meta, region=1588230740 2023-05-10 04:54:00,484 INFO [Listener at localhost/45007] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683694470484 2023-05-10 04:54:00,486 DEBUG [Listener at localhost/45007] assignment.AssignmentManager(1517): Joining cluster... 2023-05-10 04:54:00,486 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:54:00,488 DEBUG [Listener at localhost/45007] assignment.RegionStateStore(147): Load hbase:meta entry region=ccfda81dc2f13ffc1429bef67d349028, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36665,1683694419452, regionLocation=jenkins-hbase4.apache.org,36665,1683694419452, openSeqNum=2 2023-05-10 04:54:00,488 DEBUG [Listener at localhost/45007] assignment.RegionStateStore(147): Load hbase:meta entry region=4dcb62249031b560a1bc5d03e43ac855, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,37823,1683694419534, regionLocation=jenkins-hbase4.apache.org,37823,1683694419534, openSeqNum=2 2023-05-10 04:54:00,488 INFO [Listener at localhost/45007] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-10 04:54:00,488 INFO [Listener at localhost/45007] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683694500488 2023-05-10 04:54:00,488 INFO [Listener at localhost/45007] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683694560488 2023-05-10 04:54:00,488 INFO [Listener at localhost/45007] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-05-10 04:54:00,489 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-10 04:54:00,491 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-10 04:54:00,491 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-05-10 04:54:00,491 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-10 04:54:00,491 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-10 04:54:00,739 INFO [Listener at localhost/45007] procedure.MasterProcedureTestingUtility(509): Restart 3 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:54:00,739 INFO [Listener at localhost/45007] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-10 04:54:00,739 INFO [Listener at localhost/45007] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-10 04:54:00,739 INFO [Listener at localhost/45007] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-10 04:54:00,740 ERROR [Listener at localhost/45007] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-10 04:54:00,740 INFO [Listener at localhost/45007] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-10 04:54:00,740 INFO [Listener at localhost/45007] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-10 04:54:00,740 INFO [Listener at localhost/45007] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-10 04:54:00,740 INFO [Listener at localhost/45007] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-10 04:54:00,741 INFO [Listener at localhost/45007] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-10 04:54:00,742 DEBUG [Listener at localhost/45007] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-05-10 04:54:00,742 DEBUG [Listener at localhost/45007] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-05-10 04:54:00,742 DEBUG [Listener at localhost/45007] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-05-10 04:54:00,742 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-10 04:54:00,742 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-10 04:54:00,743 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-10 04:54:00,743 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-10 04:54:00,743 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:54:00,743 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=93c2cc329b1fe1893cfbc28c56960522, daughterA=19040b6538a6bdeed7c833cfb44614b6, daughterB=054f69c06f126aad97b290410d5d6188 2023-05-10 04:54:00,743 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:54:00,743 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:54:00,744 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-10 04:54:00,744 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6c70954228a0bbe97dfdc76861367c66, daughterA=75e2592cb2ac08bd9f617bb9d0a0c896, daughterB=b90fd344aff14f0728a80c3d09539e54 2023-05-10 04:54:00,744 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-05-10 04:54:00,744 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-10 04:54:00,744 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-10 04:54:00,745 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:54:00,745 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=4dcb62249031b560a1bc5d03e43ac855, UNASSIGN 2023-05-10 04:54:00,745 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 4dcb62249031b560a1bc5d03e43ac855, server=jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:54:00,745 DEBUG [Listener at localhost/45007] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 held the lock before restarting, call acquireLock to restore it. 2023-05-10 04:54:00,745 INFO [Listener at localhost/45007] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:54:00,745 INFO [Listener at localhost/45007] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:54:00,745 INFO [Listener at localhost/45007] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:54:00,745 DEBUG [Listener at localhost/45007] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=4dcb62249031b560a1bc5d03e43ac855, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-05-10 04:54:00,745 DEBUG [Listener at localhost/45007] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 4dcb62249031b560a1bc5d03e43ac855, server=jenkins-hbase4.apache.org,37823,1683694419534 didn't hold the lock before restarting, skip acquiring lock. 2023-05-10 04:54:00,745 INFO [Listener at localhost/45007] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-05-10 04:54:00,746 DEBUG [Listener at localhost/45007] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-10 04:54:00,746 DEBUG [Listener at localhost/45007] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,37995,1683694419499, table=hbase:meta, region=1588230740 2023-05-10 04:54:00,747 INFO [Listener at localhost/45007] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683694470747 2023-05-10 04:54:00,748 DEBUG [Listener at localhost/45007] assignment.AssignmentManager(1517): Joining cluster... 2023-05-10 04:54:00,748 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:54:00,750 DEBUG [Listener at localhost/45007] assignment.RegionStateStore(147): Load hbase:meta entry region=ccfda81dc2f13ffc1429bef67d349028, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36665,1683694419452, regionLocation=jenkins-hbase4.apache.org,36665,1683694419452, openSeqNum=2 2023-05-10 04:54:00,751 DEBUG [Listener at localhost/45007] assignment.RegionStateStore(147): Load hbase:meta entry region=4dcb62249031b560a1bc5d03e43ac855, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,37823,1683694419534, regionLocation=jenkins-hbase4.apache.org,37823,1683694419534, openSeqNum=2 2023-05-10 04:54:00,751 INFO [Listener at localhost/45007] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-10 04:54:00,751 INFO [Listener at localhost/45007] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683694500751 2023-05-10 04:54:00,751 INFO [Listener at localhost/45007] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683694560751 2023-05-10 04:54:00,751 INFO [Listener at localhost/45007] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-05-10 04:54:00,751 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-10 04:54:00,754 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1054): Stored [pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=4dcb62249031b560a1bc5d03e43ac855, ASSIGN] 2023-05-10 04:54:00,754 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-10 04:54:00,754 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-05-10 04:54:00,754 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-10 04:54:00,754 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-10 04:54:01,001 INFO [Listener at localhost/45007] procedure.MasterProcedureTestingUtility(509): Restart 4 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:54:01,001 INFO [Listener at localhost/45007] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-10 04:54:01,001 INFO [Listener at localhost/45007] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-10 04:54:01,001 INFO [Listener at localhost/45007] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-10 04:54:01,003 ERROR [Listener at localhost/45007] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-10 04:54:01,003 INFO [Listener at localhost/45007] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-10 04:54:01,003 INFO [Listener at localhost/45007] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-10 04:54:01,003 INFO [Listener at localhost/45007] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-10 04:54:01,003 INFO [Listener at localhost/45007] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-10 04:54:01,004 INFO [Listener at localhost/45007] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-10 04:54:01,005 DEBUG [Listener at localhost/45007] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-05-10 04:54:01,005 DEBUG [Listener at localhost/45007] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-05-10 04:54:01,005 DEBUG [Listener at localhost/45007] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-05-10 04:54:01,006 DEBUG [Listener at localhost/45007] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-05-10 04:54:01,006 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-10 04:54:01,006 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-10 04:54:01,007 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-10 04:54:01,007 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-10 04:54:01,007 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:54:01,007 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=93c2cc329b1fe1893cfbc28c56960522, daughterA=19040b6538a6bdeed7c833cfb44614b6, daughterB=054f69c06f126aad97b290410d5d6188 2023-05-10 04:54:01,008 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:54:01,008 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:54:01,008 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-10 04:54:01,009 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6c70954228a0bbe97dfdc76861367c66, daughterA=75e2592cb2ac08bd9f617bb9d0a0c896, daughterB=b90fd344aff14f0728a80c3d09539e54 2023-05-10 04:54:01,009 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-05-10 04:54:01,009 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-10 04:54:01,009 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-10 04:54:01,011 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:54:01,011 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=4dcb62249031b560a1bc5d03e43ac855, UNASSIGN 2023-05-10 04:54:01,011 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 4dcb62249031b560a1bc5d03e43ac855, server=jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:54:01,011 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=4dcb62249031b560a1bc5d03e43ac855, ASSIGN 2023-05-10 04:54:01,011 DEBUG [Listener at localhost/45007] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 held the lock before restarting, call acquireLock to restore it. 2023-05-10 04:54:01,011 INFO [Listener at localhost/45007] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:54:01,011 INFO [Listener at localhost/45007] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:54:01,011 INFO [Listener at localhost/45007] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:54:01,012 DEBUG [Listener at localhost/45007] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=4dcb62249031b560a1bc5d03e43ac855, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-05-10 04:54:01,012 DEBUG [Listener at localhost/45007] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 4dcb62249031b560a1bc5d03e43ac855, server=jenkins-hbase4.apache.org,37823,1683694419534 didn't hold the lock before restarting, skip acquiring lock. 2023-05-10 04:54:01,012 DEBUG [Listener at localhost/45007] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=4dcb62249031b560a1bc5d03e43ac855, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-05-10 04:54:01,012 INFO [Listener at localhost/45007] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-05-10 04:54:01,013 DEBUG [Listener at localhost/45007] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-10 04:54:01,013 DEBUG [Listener at localhost/45007] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,37995,1683694419499, table=hbase:meta, region=1588230740 2023-05-10 04:54:01,013 INFO [Listener at localhost/45007] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=4dcb62249031b560a1bc5d03e43ac855, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=4dcb62249031b560a1bc5d03e43ac855 to restore RIT 2023-05-10 04:54:01,013 INFO [Listener at localhost/45007] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683694471013 2023-05-10 04:54:01,015 DEBUG [Listener at localhost/45007] assignment.AssignmentManager(1517): Joining cluster... 2023-05-10 04:54:01,015 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:54:01,016 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=4dcb62249031b560a1bc5d03e43ac855, ASSIGN 2023-05-10 04:54:01,018 DEBUG [Listener at localhost/45007] assignment.RegionStateStore(147): Load hbase:meta entry region=ccfda81dc2f13ffc1429bef67d349028, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36665,1683694419452, regionLocation=jenkins-hbase4.apache.org,36665,1683694419452, openSeqNum=2 2023-05-10 04:54:01,019 DEBUG [Listener at localhost/45007] assignment.RegionStateStore(147): Load hbase:meta entry region=4dcb62249031b560a1bc5d03e43ac855, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,37823,1683694419534, regionLocation=jenkins-hbase4.apache.org,37823,1683694419534, openSeqNum=2 2023-05-10 04:54:01,019 INFO [Listener at localhost/45007] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-10 04:54:01,019 INFO [Listener at localhost/45007] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683694501019 2023-05-10 04:54:01,019 INFO [Listener at localhost/45007] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683694561019 2023-05-10 04:54:01,019 INFO [Listener at localhost/45007] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-05-10 04:54:01,019 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-10 04:54:01,024 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-10 04:54:01,024 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-05-10 04:54:01,024 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-10 04:54:01,024 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-10 04:54:01,269 INFO [Listener at localhost/45007] procedure.MasterProcedureTestingUtility(509): Restart 5 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:54:01,269 INFO [Listener at localhost/45007] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-10 04:54:01,269 INFO [Listener at localhost/45007] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-10 04:54:01,270 INFO [Listener at localhost/45007] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-10 04:54:01,272 ERROR [Listener at localhost/45007] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-10 04:54:01,272 INFO [Listener at localhost/45007] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-10 04:54:01,272 INFO [Listener at localhost/45007] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-10 04:54:01,272 INFO [Listener at localhost/45007] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-10 04:54:01,272 INFO [Listener at localhost/45007] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-10 04:54:01,273 INFO [Listener at localhost/45007] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-10 04:54:01,274 DEBUG [Listener at localhost/45007] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-05-10 04:54:01,275 DEBUG [Listener at localhost/45007] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-05-10 04:54:01,275 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-10 04:54:01,275 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-10 04:54:01,275 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-10 04:54:01,276 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-10 04:54:01,276 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:54:01,276 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=93c2cc329b1fe1893cfbc28c56960522, daughterA=19040b6538a6bdeed7c833cfb44614b6, daughterB=054f69c06f126aad97b290410d5d6188 2023-05-10 04:54:01,277 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:54:01,277 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:54:01,277 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-10 04:54:01,278 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6c70954228a0bbe97dfdc76861367c66, daughterA=75e2592cb2ac08bd9f617bb9d0a0c896, daughterB=b90fd344aff14f0728a80c3d09539e54 2023-05-10 04:54:01,278 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-05-10 04:54:01,278 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-10 04:54:01,278 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-10 04:54:01,279 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:54:01,279 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=4dcb62249031b560a1bc5d03e43ac855, ASSIGN 2023-05-10 04:54:01,279 DEBUG [Listener at localhost/45007] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 held the lock before restarting, call acquireLock to restore it. 2023-05-10 04:54:01,279 INFO [Listener at localhost/45007] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:54:01,279 INFO [Listener at localhost/45007] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:54:01,279 INFO [Listener at localhost/45007] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:54:01,279 DEBUG [Listener at localhost/45007] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=4dcb62249031b560a1bc5d03e43ac855, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-05-10 04:54:01,280 INFO [Listener at localhost/45007] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-05-10 04:54:01,280 DEBUG [Listener at localhost/45007] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-10 04:54:01,281 DEBUG [Listener at localhost/45007] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,37995,1683694419499, table=hbase:meta, region=1588230740 2023-05-10 04:54:01,281 INFO [Listener at localhost/45007] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=4dcb62249031b560a1bc5d03e43ac855, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=4dcb62249031b560a1bc5d03e43ac855 to restore RIT 2023-05-10 04:54:01,281 INFO [Listener at localhost/45007] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683694471281 2023-05-10 04:54:01,283 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:54:01,283 DEBUG [Listener at localhost/45007] assignment.AssignmentManager(1517): Joining cluster... 2023-05-10 04:54:01,284 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=4dcb62249031b560a1bc5d03e43ac855, ASSIGN 2023-05-10 04:54:01,286 DEBUG [Listener at localhost/45007] assignment.RegionStateStore(147): Load hbase:meta entry region=ccfda81dc2f13ffc1429bef67d349028, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36665,1683694419452, regionLocation=jenkins-hbase4.apache.org,36665,1683694419452, openSeqNum=2 2023-05-10 04:54:01,286 DEBUG [Listener at localhost/45007] assignment.RegionStateStore(147): Load hbase:meta entry region=4dcb62249031b560a1bc5d03e43ac855, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,37823,1683694419534, regionLocation=jenkins-hbase4.apache.org,37823,1683694419534, openSeqNum=2 2023-05-10 04:54:01,286 INFO [Listener at localhost/45007] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-10 04:54:01,286 INFO [Listener at localhost/45007] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683694501286 2023-05-10 04:54:01,287 INFO [Listener at localhost/45007] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683694561287 2023-05-10 04:54:01,287 INFO [Listener at localhost/45007] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-05-10 04:54:01,287 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-10 04:54:01,289 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-10 04:54:01,289 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-05-10 04:54:01,289 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-10 04:54:01,289 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-10 04:54:01,537 INFO [Listener at localhost/45007] procedure.MasterProcedureTestingUtility(509): Restart 6 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:54:01,537 INFO [Listener at localhost/45007] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-10 04:54:01,537 INFO [Listener at localhost/45007] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-10 04:54:01,537 INFO [Listener at localhost/45007] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-10 04:54:01,538 ERROR [Listener at localhost/45007] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-10 04:54:01,538 INFO [Listener at localhost/45007] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-10 04:54:01,538 INFO [Listener at localhost/45007] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-10 04:54:01,538 INFO [Listener at localhost/45007] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-10 04:54:01,538 INFO [Listener at localhost/45007] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-10 04:54:01,539 INFO [Listener at localhost/45007] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-10 04:54:01,540 DEBUG [Listener at localhost/45007] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-05-10 04:54:01,540 DEBUG [Listener at localhost/45007] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-05-10 04:54:01,540 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-10 04:54:01,541 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-10 04:54:01,541 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-10 04:54:01,541 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-10 04:54:01,542 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:54:01,542 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=93c2cc329b1fe1893cfbc28c56960522, daughterA=19040b6538a6bdeed7c833cfb44614b6, daughterB=054f69c06f126aad97b290410d5d6188 2023-05-10 04:54:01,542 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:54:01,542 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:54:01,543 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-10 04:54:01,543 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6c70954228a0bbe97dfdc76861367c66, daughterA=75e2592cb2ac08bd9f617bb9d0a0c896, daughterB=b90fd344aff14f0728a80c3d09539e54 2023-05-10 04:54:01,543 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-05-10 04:54:01,544 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-10 04:54:01,544 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-10 04:54:01,544 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:54:01,544 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=4dcb62249031b560a1bc5d03e43ac855, ASSIGN 2023-05-10 04:54:01,545 DEBUG [Listener at localhost/45007] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 held the lock before restarting, call acquireLock to restore it. 2023-05-10 04:54:01,545 INFO [Listener at localhost/45007] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:54:01,545 INFO [Listener at localhost/45007] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:54:01,545 INFO [Listener at localhost/45007] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:54:01,545 DEBUG [Listener at localhost/45007] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=4dcb62249031b560a1bc5d03e43ac855, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-05-10 04:54:01,545 INFO [Listener at localhost/45007] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-05-10 04:54:01,546 DEBUG [Listener at localhost/45007] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-10 04:54:01,546 DEBUG [Listener at localhost/45007] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,37995,1683694419499, table=hbase:meta, region=1588230740 2023-05-10 04:54:01,547 INFO [Listener at localhost/45007] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=4dcb62249031b560a1bc5d03e43ac855, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=4dcb62249031b560a1bc5d03e43ac855 to restore RIT 2023-05-10 04:54:01,547 INFO [Listener at localhost/45007] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683694471547 2023-05-10 04:54:01,548 DEBUG [Listener at localhost/45007] assignment.AssignmentManager(1517): Joining cluster... 2023-05-10 04:54:01,549 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:54:01,550 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=4dcb62249031b560a1bc5d03e43ac855, ASSIGN 2023-05-10 04:54:01,551 DEBUG [Listener at localhost/45007] assignment.RegionStateStore(147): Load hbase:meta entry region=ccfda81dc2f13ffc1429bef67d349028, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36665,1683694419452, regionLocation=jenkins-hbase4.apache.org,36665,1683694419452, openSeqNum=2 2023-05-10 04:54:01,552 DEBUG [Listener at localhost/45007] assignment.RegionStateStore(147): Load hbase:meta entry region=4dcb62249031b560a1bc5d03e43ac855, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,37823,1683694419534, regionLocation=jenkins-hbase4.apache.org,37823,1683694419534, openSeqNum=2 2023-05-10 04:54:01,552 INFO [Listener at localhost/45007] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-10 04:54:01,552 INFO [Listener at localhost/45007] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683694501552 2023-05-10 04:54:01,552 INFO [Listener at localhost/45007] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683694561552 2023-05-10 04:54:01,552 INFO [Listener at localhost/45007] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-05-10 04:54:01,552 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-10 04:54:01,554 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-10 04:54:01,554 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-05-10 04:54:01,554 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-10 04:54:01,554 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-10 04:54:01,802 INFO [Listener at localhost/45007] procedure.MasterProcedureTestingUtility(509): Restart 7 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:54:01,802 INFO [Listener at localhost/45007] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-10 04:54:01,802 INFO [Listener at localhost/45007] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-10 04:54:01,802 INFO [Listener at localhost/45007] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-10 04:54:01,803 ERROR [Listener at localhost/45007] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-10 04:54:01,803 INFO [Listener at localhost/45007] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-10 04:54:01,803 INFO [Listener at localhost/45007] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-10 04:54:01,803 INFO [Listener at localhost/45007] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-10 04:54:01,803 INFO [Listener at localhost/45007] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-10 04:54:01,804 INFO [Listener at localhost/45007] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-10 04:54:01,804 DEBUG [Listener at localhost/45007] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-05-10 04:54:01,805 DEBUG [Listener at localhost/45007] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-05-10 04:54:01,805 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-10 04:54:01,805 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-10 04:54:01,805 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-10 04:54:01,805 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-10 04:54:01,806 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:54:01,806 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=93c2cc329b1fe1893cfbc28c56960522, daughterA=19040b6538a6bdeed7c833cfb44614b6, daughterB=054f69c06f126aad97b290410d5d6188 2023-05-10 04:54:01,806 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:54:01,806 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:54:01,806 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-10 04:54:01,807 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6c70954228a0bbe97dfdc76861367c66, daughterA=75e2592cb2ac08bd9f617bb9d0a0c896, daughterB=b90fd344aff14f0728a80c3d09539e54 2023-05-10 04:54:01,807 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-05-10 04:54:01,807 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-10 04:54:01,807 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-10 04:54:01,807 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:54:01,808 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=4dcb62249031b560a1bc5d03e43ac855, ASSIGN 2023-05-10 04:54:01,808 DEBUG [Listener at localhost/45007] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 held the lock before restarting, call acquireLock to restore it. 2023-05-10 04:54:01,808 INFO [Listener at localhost/45007] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:54:01,808 INFO [Listener at localhost/45007] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:54:01,808 INFO [Listener at localhost/45007] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:54:01,808 DEBUG [Listener at localhost/45007] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=4dcb62249031b560a1bc5d03e43ac855, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-05-10 04:54:01,808 INFO [Listener at localhost/45007] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-05-10 04:54:01,809 DEBUG [Listener at localhost/45007] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-10 04:54:01,809 DEBUG [Listener at localhost/45007] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,37995,1683694419499, table=hbase:meta, region=1588230740 2023-05-10 04:54:01,809 INFO [Listener at localhost/45007] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=4dcb62249031b560a1bc5d03e43ac855, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=4dcb62249031b560a1bc5d03e43ac855 to restore RIT 2023-05-10 04:54:01,810 INFO [Listener at localhost/45007] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683694471810 2023-05-10 04:54:01,810 DEBUG [Listener at localhost/45007] assignment.AssignmentManager(1517): Joining cluster... 2023-05-10 04:54:01,812 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:54:01,814 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=4dcb62249031b560a1bc5d03e43ac855, ASSIGN 2023-05-10 04:54:01,816 DEBUG [Listener at localhost/45007] assignment.RegionStateStore(147): Load hbase:meta entry region=ccfda81dc2f13ffc1429bef67d349028, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36665,1683694419452, regionLocation=jenkins-hbase4.apache.org,36665,1683694419452, openSeqNum=2 2023-05-10 04:54:01,816 DEBUG [Listener at localhost/45007] assignment.RegionStateStore(147): Load hbase:meta entry region=4dcb62249031b560a1bc5d03e43ac855, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,37823,1683694419534, regionLocation=jenkins-hbase4.apache.org,37823,1683694419534, openSeqNum=2 2023-05-10 04:54:01,816 INFO [Listener at localhost/45007] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-10 04:54:01,816 INFO [Listener at localhost/45007] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683694501816 2023-05-10 04:54:01,816 INFO [Listener at localhost/45007] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683694561816 2023-05-10 04:54:01,816 INFO [Listener at localhost/45007] assignment.AssignmentManager(1536): Joined the cluster in 6 msec 2023-05-10 04:54:01,816 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-10 04:54:01,819 INFO [PEWorker-1] procedure2.ProcedureExecutor(1528): Rolled back pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 exec-time=5.0210 sec 2023-05-10 04:54:01,819 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=4dcb62249031b560a1bc5d03e43ac855, ASSIGN 2023-05-10 04:54:01,820 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=4dcb62249031b560a1bc5d03e43ac855, ASSIGN; state=CLOSED, location=jenkins-hbase4.apache.org,37823,1683694419534; forceNewPlan=false, retain=true 2023-05-10 04:54:01,970 INFO [jenkins-hbase4:37903] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-05-10 04:54:01,971 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=47 updating hbase:meta row=4dcb62249031b560a1bc5d03e43ac855, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:54:01,971 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1683694434423.4dcb62249031b560a1bc5d03e43ac855.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1683694441971"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683694441971"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683694441971"}]},"ts":"1683694441971"} 2023-05-10 04:54:01,973 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=48, ppid=47, state=RUNNABLE; OpenRegionProcedure 4dcb62249031b560a1bc5d03e43ac855, server=jenkins-hbase4.apache.org,37823,1683694419534}] 2023-05-10 04:54:01,973 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-10 04:54:01,973 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=47, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_OPENED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=4dcb62249031b560a1bc5d03e43ac855, ASSIGN 2023-05-10 04:54:01,973 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-10 04:54:01,973 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=47, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_OPENED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=4dcb62249031b560a1bc5d03e43ac855, ASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-10 04:54:02,066 WARN [Listener at localhost/45007] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-05-10 04:54:02,067 WARN [Listener at localhost/45007] procedure.MasterProcedureTestingUtility(526): ProcedureExecutor not running, may have been stopped by pending procedure due to KillAndToggleBeforeStoreUpdate flag. 2023-05-10 04:54:02,067 INFO [Listener at localhost/45007] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-10 04:54:02,067 INFO [Listener at localhost/45007] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-10 04:54:02,067 INFO [Listener at localhost/45007] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-10 04:54:02,067 ERROR [Listener at localhost/45007] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-10 04:54:02,067 INFO [Listener at localhost/45007] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-10 04:54:02,067 INFO [Listener at localhost/45007] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-10 04:54:02,067 INFO [Listener at localhost/45007] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-10 04:54:02,067 INFO [Listener at localhost/45007] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-10 04:54:02,068 INFO [Listener at localhost/45007] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-10 04:54:02,069 DEBUG [Listener at localhost/45007] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[0] 2023-05-10 04:54:02,069 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-10 04:54:02,069 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-10 04:54:02,070 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-10 04:54:02,070 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-10 04:54:02,070 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:54:02,070 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=93c2cc329b1fe1893cfbc28c56960522, daughterA=19040b6538a6bdeed7c833cfb44614b6, daughterB=054f69c06f126aad97b290410d5d6188 2023-05-10 04:54:02,070 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:54:02,070 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:54:02,071 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-10 04:54:02,071 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6c70954228a0bbe97dfdc76861367c66, daughterA=75e2592cb2ac08bd9f617bb9d0a0c896, daughterB=b90fd344aff14f0728a80c3d09539e54 2023-05-10 04:54:02,071 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-05-10 04:54:02,071 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-10 04:54:02,071 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-10 04:54:02,071 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:54:02,072 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=4dcb62249031b560a1bc5d03e43ac855, ASSIGN 2023-05-10 04:54:02,072 DEBUG [Listener at localhost/45007] procedure2.Procedure(966): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=4dcb62249031b560a1bc5d03e43ac855, ASSIGN held the lock before restarting, call acquireLock to restore it. 2023-05-10 04:54:02,072 INFO [Listener at localhost/45007] procedure.MasterProcedureScheduler(727): Took xlock for pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=4dcb62249031b560a1bc5d03e43ac855, ASSIGN 2023-05-10 04:54:02,072 INFO [Listener at localhost/45007] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 3 msec 2023-05-10 04:54:02,072 DEBUG [Listener at localhost/45007] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-10 04:54:02,073 DEBUG [Listener at localhost/45007] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,37995,1683694419499, table=hbase:meta, region=1588230740 2023-05-10 04:54:02,073 INFO [Listener at localhost/45007] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=4dcb62249031b560a1bc5d03e43ac855, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=4dcb62249031b560a1bc5d03e43ac855 to restore RIT 2023-05-10 04:54:02,083 INFO [Listener at localhost/45007] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683694472083 2023-05-10 04:54:02,084 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=4dcb62249031b560a1bc5d03e43ac855, ASSIGN 2023-05-10 04:54:02,084 DEBUG [Listener at localhost/45007] assignment.AssignmentManager(1517): Joining cluster... 2023-05-10 04:54:02,086 DEBUG [Listener at localhost/45007] assignment.RegionStateStore(147): Load hbase:meta entry region=ccfda81dc2f13ffc1429bef67d349028, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36665,1683694419452, regionLocation=jenkins-hbase4.apache.org,36665,1683694419452, openSeqNum=2 2023-05-10 04:54:02,086 DEBUG [Listener at localhost/45007] assignment.RegionStateStore(147): Load hbase:meta entry region=4dcb62249031b560a1bc5d03e43ac855, regionState=OPENING, lastHost=jenkins-hbase4.apache.org,37823,1683694419534, regionLocation=jenkins-hbase4.apache.org,37823,1683694419534, openSeqNum=2 2023-05-10 04:54:02,086 INFO [Listener at localhost/45007] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-10 04:54:02,086 INFO [Listener at localhost/45007] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683694502086 2023-05-10 04:54:02,086 INFO [Listener at localhost/45007] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683694562086 2023-05-10 04:54:02,086 INFO [Listener at localhost/45007] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-05-10 04:54:02,086 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=47 updating hbase:meta row=4dcb62249031b560a1bc5d03e43ac855, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:54:02,087 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1683694434423.4dcb62249031b560a1bc5d03e43ac855.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1683694442086"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683694442086"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683694442086"}]},"ts":"1683694442086"} 2023-05-10 04:54:02,088 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=48, ppid=47, state=RUNNABLE; OpenRegionProcedure 4dcb62249031b560a1bc5d03e43ac855, server=jenkins-hbase4.apache.org,37823,1683694419534}] 2023-05-10 04:54:02,243 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testRollbackAndDoubleExecution,,1683694434423.4dcb62249031b560a1bc5d03e43ac855. 2023-05-10 04:54:02,244 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 4dcb62249031b560a1bc5d03e43ac855, NAME => 'testRollbackAndDoubleExecution,,1683694434423.4dcb62249031b560a1bc5d03e43ac855.', STARTKEY => '', ENDKEY => ''} 2023-05-10 04:54:02,244 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-05-10 04:54:02,244 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackAndDoubleExecution 4dcb62249031b560a1bc5d03e43ac855 2023-05-10 04:54:02,244 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testRollbackAndDoubleExecution,,1683694434423.4dcb62249031b560a1bc5d03e43ac855.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-10 04:54:02,244 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 4dcb62249031b560a1bc5d03e43ac855 2023-05-10 04:54:02,244 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 4dcb62249031b560a1bc5d03e43ac855 2023-05-10 04:54:02,245 INFO [StoreOpener-4dcb62249031b560a1bc5d03e43ac855-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 4dcb62249031b560a1bc5d03e43ac855 2023-05-10 04:54:02,246 DEBUG [StoreOpener-4dcb62249031b560a1bc5d03e43ac855-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf1 2023-05-10 04:54:02,247 DEBUG [StoreOpener-4dcb62249031b560a1bc5d03e43ac855-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf1 2023-05-10 04:54:02,247 INFO [StoreOpener-4dcb62249031b560a1bc5d03e43ac855-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 4dcb62249031b560a1bc5d03e43ac855 columnFamilyName cf1 2023-05-10 04:54:02,253 DEBUG [StoreOpener-4dcb62249031b560a1bc5d03e43ac855-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf1/118eb6964a944d8bb1efaa8c1933daa3 2023-05-10 04:54:02,256 DEBUG [StoreOpener-4dcb62249031b560a1bc5d03e43ac855-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf1/439486fbe6944c55ac7e2e681288c043 2023-05-10 04:54:02,258 DEBUG [StoreOpener-4dcb62249031b560a1bc5d03e43ac855-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf1/6250e2688ebe49cda50b3b8449609ff3 2023-05-10 04:54:02,261 DEBUG [StoreOpener-4dcb62249031b560a1bc5d03e43ac855-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf1/8df5350bb6b84fedabe8af09648762f1 2023-05-10 04:54:02,264 DEBUG [StoreOpener-4dcb62249031b560a1bc5d03e43ac855-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf1/b348d048ffd647f0b322ec445c520c0e 2023-05-10 04:54:02,267 DEBUG [StoreOpener-4dcb62249031b560a1bc5d03e43ac855-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf1/be588b1c116843cb87e38baf82c3a2db 2023-05-10 04:54:02,270 DEBUG [StoreOpener-4dcb62249031b560a1bc5d03e43ac855-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf1/fc0d292a16e34289b813715b5a9e56bf 2023-05-10 04:54:02,270 INFO [StoreOpener-4dcb62249031b560a1bc5d03e43ac855-1] regionserver.HStore(310): Store=4dcb62249031b560a1bc5d03e43ac855/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-10 04:54:02,270 INFO [StoreOpener-4dcb62249031b560a1bc5d03e43ac855-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 4dcb62249031b560a1bc5d03e43ac855 2023-05-10 04:54:02,271 DEBUG [StoreOpener-4dcb62249031b560a1bc5d03e43ac855-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf2 2023-05-10 04:54:02,271 DEBUG [StoreOpener-4dcb62249031b560a1bc5d03e43ac855-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf2 2023-05-10 04:54:02,271 INFO [StoreOpener-4dcb62249031b560a1bc5d03e43ac855-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 4dcb62249031b560a1bc5d03e43ac855 columnFamilyName cf2 2023-05-10 04:54:02,278 DEBUG [StoreOpener-4dcb62249031b560a1bc5d03e43ac855-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf2/11c6828a4683438fba9bf4cd76081c5d 2023-05-10 04:54:02,281 DEBUG [StoreOpener-4dcb62249031b560a1bc5d03e43ac855-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf2/3d1fe957545445a4befe08b6b30a80dc 2023-05-10 04:54:02,284 DEBUG [StoreOpener-4dcb62249031b560a1bc5d03e43ac855-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf2/50823f0b97364b55999bb34d2161b4b9 2023-05-10 04:54:02,287 DEBUG [StoreOpener-4dcb62249031b560a1bc5d03e43ac855-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf2/81fa4aef7d904ef1a24ea0776a615d6f 2023-05-10 04:54:02,292 DEBUG [StoreOpener-4dcb62249031b560a1bc5d03e43ac855-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf2/c543f3f56d754808a11f335a526021df 2023-05-10 04:54:02,295 DEBUG [StoreOpener-4dcb62249031b560a1bc5d03e43ac855-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf2/d22e75e4a5124b3299106ffa2fc21709 2023-05-10 04:54:02,298 DEBUG [StoreOpener-4dcb62249031b560a1bc5d03e43ac855-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf2/d89cd6883f824b51afd167aaa88a54be 2023-05-10 04:54:02,298 INFO [StoreOpener-4dcb62249031b560a1bc5d03e43ac855-1] regionserver.HStore(310): Store=4dcb62249031b560a1bc5d03e43ac855/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-10 04:54:02,299 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855 2023-05-10 04:54:02,300 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855 2023-05-10 04:54:02,302 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(602): Cleaned up old failed split transaction detritus: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/.splits 2023-05-10 04:54:02,302 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRollbackAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-05-10 04:54:02,304 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 4dcb62249031b560a1bc5d03e43ac855 2023-05-10 04:54:02,304 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 4dcb62249031b560a1bc5d03e43ac855; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67618981, jitterRate=0.007601335644721985}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-05-10 04:54:02,304 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 4dcb62249031b560a1bc5d03e43ac855: 2023-05-10 04:54:02,305 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2334): Post open deploy tasks for testRollbackAndDoubleExecution,,1683694434423.4dcb62249031b560a1bc5d03e43ac855., pid=48, masterSystemTime=1683694442241 2023-05-10 04:54:02,305 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-05-10 04:54:02,306 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-10 04:54:02,306 DEBUG [RS:2;jenkins-hbase4:37823-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-05-10 04:54:02,306 DEBUG [RS:2;jenkins-hbase4:37823-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-05-10 04:54:02,307 DEBUG [RS:2;jenkins-hbase4:37823-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-05-10 04:54:02,307 DEBUG [RS:2;jenkins-hbase4:37823-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-05-10 04:54:02,307 DEBUG [RS:2;jenkins-hbase4:37823-longCompactions-0] regionserver.HStore(1912): 4dcb62249031b560a1bc5d03e43ac855/cf2 is initiating minor compaction (all files) 2023-05-10 04:54:02,307 DEBUG [RS:2;jenkins-hbase4:37823-shortCompactions-0] regionserver.HStore(1912): 4dcb62249031b560a1bc5d03e43ac855/cf1 is initiating minor compaction (all files) 2023-05-10 04:54:02,308 INFO [RS:2;jenkins-hbase4:37823-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 4dcb62249031b560a1bc5d03e43ac855/cf1 in testRollbackAndDoubleExecution,,1683694434423.4dcb62249031b560a1bc5d03e43ac855. 2023-05-10 04:54:02,308 INFO [RS:2;jenkins-hbase4:37823-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 4dcb62249031b560a1bc5d03e43ac855/cf2 in testRollbackAndDoubleExecution,,1683694434423.4dcb62249031b560a1bc5d03e43ac855. 2023-05-10 04:54:02,308 INFO [RS:2;jenkins-hbase4:37823-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf2/d22e75e4a5124b3299106ffa2fc21709, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf2/11c6828a4683438fba9bf4cd76081c5d, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf2/3d1fe957545445a4befe08b6b30a80dc, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf2/50823f0b97364b55999bb34d2161b4b9, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf2/d89cd6883f824b51afd167aaa88a54be, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf2/c543f3f56d754808a11f335a526021df, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf2/81fa4aef7d904ef1a24ea0776a615d6f] into tmpdir=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/.tmp, totalSize=34.9 K 2023-05-10 04:54:02,308 INFO [RS:2;jenkins-hbase4:37823-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf1/be588b1c116843cb87e38baf82c3a2db, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf1/fc0d292a16e34289b813715b5a9e56bf, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf1/439486fbe6944c55ac7e2e681288c043, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf1/8df5350bb6b84fedabe8af09648762f1, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf1/b348d048ffd647f0b322ec445c520c0e, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf1/118eb6964a944d8bb1efaa8c1933daa3, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf1/6250e2688ebe49cda50b3b8449609ff3] into tmpdir=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/.tmp, totalSize=34.9 K 2023-05-10 04:54:02,308 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2361): Finished post open deploy task for testRollbackAndDoubleExecution,,1683694434423.4dcb62249031b560a1bc5d03e43ac855. 2023-05-10 04:54:02,308 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testRollbackAndDoubleExecution,,1683694434423.4dcb62249031b560a1bc5d03e43ac855. 2023-05-10 04:54:02,309 DEBUG [RS:2;jenkins-hbase4:37823-longCompactions-0] compactions.Compactor(207): Compacting d22e75e4a5124b3299106ffa2fc21709, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1683694434928 2023-05-10 04:54:02,309 DEBUG [RS:2;jenkins-hbase4:37823-shortCompactions-0] compactions.Compactor(207): Compacting be588b1c116843cb87e38baf82c3a2db, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1683694434928 2023-05-10 04:54:02,309 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=47 updating hbase:meta row=4dcb62249031b560a1bc5d03e43ac855, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:54:02,309 DEBUG [RS:2;jenkins-hbase4:37823-longCompactions-0] compactions.Compactor(207): Compacting 11c6828a4683438fba9bf4cd76081c5d, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1683694435116 2023-05-10 04:54:02,309 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackAndDoubleExecution,,1683694434423.4dcb62249031b560a1bc5d03e43ac855.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1683694442309"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1683694442309"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1683694442309"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683694442309"}]},"ts":"1683694442309"} 2023-05-10 04:54:02,309 DEBUG [RS:2;jenkins-hbase4:37823-shortCompactions-0] compactions.Compactor(207): Compacting fc0d292a16e34289b813715b5a9e56bf, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1683694435116 2023-05-10 04:54:02,310 DEBUG [RS:2;jenkins-hbase4:37823-longCompactions-0] compactions.Compactor(207): Compacting 3d1fe957545445a4befe08b6b30a80dc, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1683694435720 2023-05-10 04:54:02,310 DEBUG [RS:2;jenkins-hbase4:37823-shortCompactions-0] compactions.Compactor(207): Compacting 439486fbe6944c55ac7e2e681288c043, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1683694435720 2023-05-10 04:54:02,310 DEBUG [RS:2;jenkins-hbase4:37823-longCompactions-0] compactions.Compactor(207): Compacting 50823f0b97364b55999bb34d2161b4b9, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1683694435922 2023-05-10 04:54:02,310 DEBUG [RS:2;jenkins-hbase4:37823-shortCompactions-0] compactions.Compactor(207): Compacting 8df5350bb6b84fedabe8af09648762f1, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1683694435922 2023-05-10 04:54:02,310 DEBUG [RS:2;jenkins-hbase4:37823-longCompactions-0] compactions.Compactor(207): Compacting d89cd6883f824b51afd167aaa88a54be, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1683694436120 2023-05-10 04:54:02,311 DEBUG [RS:2;jenkins-hbase4:37823-shortCompactions-0] compactions.Compactor(207): Compacting b348d048ffd647f0b322ec445c520c0e, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1683694436120 2023-05-10 04:54:02,311 DEBUG [RS:2;jenkins-hbase4:37823-longCompactions-0] compactions.Compactor(207): Compacting c543f3f56d754808a11f335a526021df, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1683694436346 2023-05-10 04:54:02,311 DEBUG [RS:2;jenkins-hbase4:37823-shortCompactions-0] compactions.Compactor(207): Compacting 118eb6964a944d8bb1efaa8c1933daa3, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1683694436346 2023-05-10 04:54:02,312 DEBUG [RS:2;jenkins-hbase4:37823-longCompactions-0] compactions.Compactor(207): Compacting 81fa4aef7d904ef1a24ea0776a615d6f, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1683694436534 2023-05-10 04:54:02,312 DEBUG [RS:2;jenkins-hbase4:37823-shortCompactions-0] compactions.Compactor(207): Compacting 6250e2688ebe49cda50b3b8449609ff3, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1683694436534 2023-05-10 04:54:02,313 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=48, resume processing ppid=47 2023-05-10 04:54:02,313 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=48, ppid=47, state=SUCCESS; OpenRegionProcedure 4dcb62249031b560a1bc5d03e43ac855, server=jenkins-hbase4.apache.org,37823,1683694419534 in 223 msec 2023-05-10 04:54:02,315 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=4dcb62249031b560a1bc5d03e43ac855, ASSIGN in 1.5600 sec 2023-05-10 04:54:02,344 INFO [RS:2;jenkins-hbase4:37823-shortCompactions-0] throttle.PressureAwareThroughputController(145): 4dcb62249031b560a1bc5d03e43ac855#cf1#compaction#51 average throughput is 1.72 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-05-10 04:54:02,344 INFO [RS:2;jenkins-hbase4:37823-longCompactions-0] throttle.PressureAwareThroughputController(145): 4dcb62249031b560a1bc5d03e43ac855#cf2#compaction#50 average throughput is 1.72 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-05-10 04:54:02,363 DEBUG [RS:2;jenkins-hbase4:37823-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/.tmp/cf2/7c3fb74d4d7446aaad01350592afaade as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf2/7c3fb74d4d7446aaad01350592afaade 2023-05-10 04:54:02,364 DEBUG [RS:2;jenkins-hbase4:37823-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/.tmp/cf1/2a6e9794981742569c081f9848075313 as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf1/2a6e9794981742569c081f9848075313 2023-05-10 04:54:02,369 INFO [RS:2;jenkins-hbase4:37823-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 4dcb62249031b560a1bc5d03e43ac855/cf2 of 4dcb62249031b560a1bc5d03e43ac855 into 7c3fb74d4d7446aaad01350592afaade(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-05-10 04:54:02,369 INFO [RS:2;jenkins-hbase4:37823-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 4dcb62249031b560a1bc5d03e43ac855/cf1 of 4dcb62249031b560a1bc5d03e43ac855 into 2a6e9794981742569c081f9848075313(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-05-10 04:54:02,369 DEBUG [RS:2;jenkins-hbase4:37823-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 4dcb62249031b560a1bc5d03e43ac855: 2023-05-10 04:54:02,369 DEBUG [RS:2;jenkins-hbase4:37823-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 4dcb62249031b560a1bc5d03e43ac855: 2023-05-10 04:54:02,369 INFO [RS:2;jenkins-hbase4:37823-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRollbackAndDoubleExecution,,1683694434423.4dcb62249031b560a1bc5d03e43ac855., storeName=4dcb62249031b560a1bc5d03e43ac855/cf2, priority=9, startTime=1683694442305; duration=0sec 2023-05-10 04:54:02,369 INFO [RS:2;jenkins-hbase4:37823-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRollbackAndDoubleExecution,,1683694434423.4dcb62249031b560a1bc5d03e43ac855., storeName=4dcb62249031b560a1bc5d03e43ac855/cf1, priority=9, startTime=1683694442305; duration=0sec 2023-05-10 04:54:02,371 DEBUG [RS:2;jenkins-hbase4:37823-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-10 04:54:02,371 DEBUG [RS:2;jenkins-hbase4:37823-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-10 04:54:02,663 INFO [Listener at localhost/45007] procedure2.ProcedureTestingUtility(335): procId=44 exception: org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested 2023-05-10 04:54:02,665 DEBUG [Listener at localhost/45007] hbase.HBaseTestingUtility(3430): Waiting until all regions of table testRollbackAndDoubleExecution get assigned. Timeout = 1500ms 2023-05-10 04:54:02,667 INFO [Listener at localhost/45007] hbase.Waiter(180): Waiting up to [1,500] milli-secs(wait.for.ratio=[1]) 2023-05-10 04:54:02,670 INFO [Listener at localhost/45007] hbase.HBaseTestingUtility(3484): All regions for table testRollbackAndDoubleExecution assigned to meta. Checking AM states. 2023-05-10 04:54:02,670 INFO [Listener at localhost/45007] hbase.Waiter(180): Waiting up to [1,500] milli-secs(wait.for.ratio=[1]) 2023-05-10 04:54:02,670 INFO [Listener at localhost/45007] hbase.HBaseTestingUtility(3504): All regions for table testRollbackAndDoubleExecution assigned. 2023-05-10 04:54:02,683 WARN [Listener at localhost/45007] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-05-10 04:54:02,685 INFO [Listener at localhost/45007] client.HBaseAdmin$15(890): Started disable of testRollbackAndDoubleExecution 2023-05-10 04:54:02,685 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testRollbackAndDoubleExecution 2023-05-10 04:54:02,686 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] procedure2.ProcedureExecutor(1029): Stored pid=49, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-05-10 04:54:02,689 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(1227): Checking to see if procedure is done pid=49 2023-05-10 04:54:02,689 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683694442689"}]},"ts":"1683694442689"} 2023-05-10 04:54:02,690 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=DISABLING in hbase:meta 2023-05-10 04:54:02,692 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testRollbackAndDoubleExecution to state=DISABLING 2023-05-10 04:54:02,692 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=50, ppid=49, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=4dcb62249031b560a1bc5d03e43ac855, UNASSIGN}] 2023-05-10 04:54:02,693 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=50, ppid=49, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=4dcb62249031b560a1bc5d03e43ac855, UNASSIGN 2023-05-10 04:54:02,694 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=50 updating hbase:meta row=4dcb62249031b560a1bc5d03e43ac855, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:54:02,694 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1683694434423.4dcb62249031b560a1bc5d03e43ac855.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1683694442694"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683694442694"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683694442694"}]},"ts":"1683694442694"} 2023-05-10 04:54:02,695 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=51, ppid=50, state=RUNNABLE; CloseRegionProcedure 4dcb62249031b560a1bc5d03e43ac855, server=jenkins-hbase4.apache.org,37823,1683694419534}] 2023-05-10 04:54:02,790 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(1227): Checking to see if procedure is done pid=49 2023-05-10 04:54:02,847 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 4dcb62249031b560a1bc5d03e43ac855 2023-05-10 04:54:02,847 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 4dcb62249031b560a1bc5d03e43ac855, disabling compactions & flushes 2023-05-10 04:54:02,847 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testRollbackAndDoubleExecution,,1683694434423.4dcb62249031b560a1bc5d03e43ac855. 2023-05-10 04:54:02,847 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRollbackAndDoubleExecution,,1683694434423.4dcb62249031b560a1bc5d03e43ac855. 2023-05-10 04:54:02,847 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testRollbackAndDoubleExecution,,1683694434423.4dcb62249031b560a1bc5d03e43ac855. after waiting 0 ms 2023-05-10 04:54:02,847 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testRollbackAndDoubleExecution,,1683694434423.4dcb62249031b560a1bc5d03e43ac855. 2023-05-10 04:54:02,848 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1683694434423.4dcb62249031b560a1bc5d03e43ac855.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf1/be588b1c116843cb87e38baf82c3a2db, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf1/fc0d292a16e34289b813715b5a9e56bf, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf1/439486fbe6944c55ac7e2e681288c043, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf1/8df5350bb6b84fedabe8af09648762f1, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf1/b348d048ffd647f0b322ec445c520c0e, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf1/118eb6964a944d8bb1efaa8c1933daa3, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf1/6250e2688ebe49cda50b3b8449609ff3] to archive 2023-05-10 04:54:02,849 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1683694434423.4dcb62249031b560a1bc5d03e43ac855.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-05-10 04:54:02,851 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1683694434423.4dcb62249031b560a1bc5d03e43ac855.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf1/be588b1c116843cb87e38baf82c3a2db to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf1/be588b1c116843cb87e38baf82c3a2db 2023-05-10 04:54:02,852 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1683694434423.4dcb62249031b560a1bc5d03e43ac855.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf1/fc0d292a16e34289b813715b5a9e56bf to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf1/fc0d292a16e34289b813715b5a9e56bf 2023-05-10 04:54:02,853 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1683694434423.4dcb62249031b560a1bc5d03e43ac855.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf1/439486fbe6944c55ac7e2e681288c043 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf1/439486fbe6944c55ac7e2e681288c043 2023-05-10 04:54:02,855 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1683694434423.4dcb62249031b560a1bc5d03e43ac855.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf1/8df5350bb6b84fedabe8af09648762f1 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf1/8df5350bb6b84fedabe8af09648762f1 2023-05-10 04:54:02,856 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1683694434423.4dcb62249031b560a1bc5d03e43ac855.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf1/b348d048ffd647f0b322ec445c520c0e to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf1/b348d048ffd647f0b322ec445c520c0e 2023-05-10 04:54:02,857 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1683694434423.4dcb62249031b560a1bc5d03e43ac855.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf1/118eb6964a944d8bb1efaa8c1933daa3 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf1/118eb6964a944d8bb1efaa8c1933daa3 2023-05-10 04:54:02,858 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1683694434423.4dcb62249031b560a1bc5d03e43ac855.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf1/6250e2688ebe49cda50b3b8449609ff3 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf1/6250e2688ebe49cda50b3b8449609ff3 2023-05-10 04:54:02,862 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1683694434423.4dcb62249031b560a1bc5d03e43ac855.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf2/d22e75e4a5124b3299106ffa2fc21709, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf2/11c6828a4683438fba9bf4cd76081c5d, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf2/3d1fe957545445a4befe08b6b30a80dc, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf2/50823f0b97364b55999bb34d2161b4b9, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf2/d89cd6883f824b51afd167aaa88a54be, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf2/c543f3f56d754808a11f335a526021df, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf2/81fa4aef7d904ef1a24ea0776a615d6f] to archive 2023-05-10 04:54:02,863 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1683694434423.4dcb62249031b560a1bc5d03e43ac855.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-05-10 04:54:02,865 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1683694434423.4dcb62249031b560a1bc5d03e43ac855.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf2/d22e75e4a5124b3299106ffa2fc21709 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf2/d22e75e4a5124b3299106ffa2fc21709 2023-05-10 04:54:02,866 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1683694434423.4dcb62249031b560a1bc5d03e43ac855.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf2/11c6828a4683438fba9bf4cd76081c5d to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf2/11c6828a4683438fba9bf4cd76081c5d 2023-05-10 04:54:02,867 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1683694434423.4dcb62249031b560a1bc5d03e43ac855.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf2/3d1fe957545445a4befe08b6b30a80dc to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf2/3d1fe957545445a4befe08b6b30a80dc 2023-05-10 04:54:02,868 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1683694434423.4dcb62249031b560a1bc5d03e43ac855.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf2/50823f0b97364b55999bb34d2161b4b9 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf2/50823f0b97364b55999bb34d2161b4b9 2023-05-10 04:54:02,870 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1683694434423.4dcb62249031b560a1bc5d03e43ac855.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf2/d89cd6883f824b51afd167aaa88a54be to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf2/d89cd6883f824b51afd167aaa88a54be 2023-05-10 04:54:02,871 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1683694434423.4dcb62249031b560a1bc5d03e43ac855.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf2/c543f3f56d754808a11f335a526021df to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf2/c543f3f56d754808a11f335a526021df 2023-05-10 04:54:02,872 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1683694434423.4dcb62249031b560a1bc5d03e43ac855.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf2/81fa4aef7d904ef1a24ea0776a615d6f to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf2/81fa4aef7d904ef1a24ea0776a615d6f 2023-05-10 04:54:02,875 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-05-10 04:54:02,876 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-10 04:54:02,876 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testRollbackAndDoubleExecution,,1683694434423.4dcb62249031b560a1bc5d03e43ac855. 2023-05-10 04:54:02,876 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 4dcb62249031b560a1bc5d03e43ac855: 2023-05-10 04:54:02,877 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 4dcb62249031b560a1bc5d03e43ac855 2023-05-10 04:54:02,878 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=50 updating hbase:meta row=4dcb62249031b560a1bc5d03e43ac855, regionState=CLOSED 2023-05-10 04:54:02,878 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackAndDoubleExecution,,1683694434423.4dcb62249031b560a1bc5d03e43ac855.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1683694442878"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683694442878"}]},"ts":"1683694442878"} 2023-05-10 04:54:02,881 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=51, resume processing ppid=50 2023-05-10 04:54:02,881 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=51, ppid=50, state=SUCCESS; CloseRegionProcedure 4dcb62249031b560a1bc5d03e43ac855, server=jenkins-hbase4.apache.org,37823,1683694419534 in 184 msec 2023-05-10 04:54:02,883 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=50, resume processing ppid=49 2023-05-10 04:54:02,883 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=50, ppid=49, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=4dcb62249031b560a1bc5d03e43ac855, UNASSIGN in 189 msec 2023-05-10 04:54:02,883 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683694442883"}]},"ts":"1683694442883"} 2023-05-10 04:54:02,884 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=DISABLED in hbase:meta 2023-05-10 04:54:02,887 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testRollbackAndDoubleExecution to state=DISABLED 2023-05-10 04:54:02,888 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution in 202 msec 2023-05-10 04:54:02,992 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(1227): Checking to see if procedure is done pid=49 2023-05-10 04:54:02,992 INFO [Listener at localhost/45007] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testRollbackAndDoubleExecution, procId: 49 completed 2023-05-10 04:54:02,993 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete testRollbackAndDoubleExecution 2023-05-10 04:54:02,994 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] procedure2.ProcedureExecutor(1029): Stored pid=52, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-05-10 04:54:02,995 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=52, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-05-10 04:54:02,996 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=52, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-05-10 04:54:02,996 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(1227): Checking to see if procedure is done pid=52 2023-05-10 04:54:03,000 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855 2023-05-10 04:54:03,001 DEBUG [HFileArchiver-4] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf1, FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf2, FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/recovered.edits] 2023-05-10 04:54:03,004 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf1/2a6e9794981742569c081f9848075313 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf1/2a6e9794981742569c081f9848075313 2023-05-10 04:54:03,006 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf2/7c3fb74d4d7446aaad01350592afaade to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/cf2/7c3fb74d4d7446aaad01350592afaade 2023-05-10 04:54:03,008 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/recovered.edits/90.seqid to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855/recovered.edits/90.seqid 2023-05-10 04:54:03,009 DEBUG [HFileArchiver-4] backup.HFileArchiver(596): Deleted hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testRollbackAndDoubleExecution/4dcb62249031b560a1bc5d03e43ac855 2023-05-10 04:54:03,009 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackAndDoubleExecution regions 2023-05-10 04:54:03,012 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=52, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-05-10 04:54:03,014 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testRollbackAndDoubleExecution from hbase:meta 2023-05-10 04:54:03,015 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testRollbackAndDoubleExecution' descriptor. 2023-05-10 04:54:03,017 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=52, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-05-10 04:54:03,017 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testRollbackAndDoubleExecution' from region states. 2023-05-10 04:54:03,017 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackAndDoubleExecution,,1683694434423.4dcb62249031b560a1bc5d03e43ac855.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1683694443017"}]},"ts":"9223372036854775807"} 2023-05-10 04:54:03,018 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 1 regions from META 2023-05-10 04:54:03,018 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 4dcb62249031b560a1bc5d03e43ac855, NAME => 'testRollbackAndDoubleExecution,,1683694434423.4dcb62249031b560a1bc5d03e43ac855.', STARTKEY => '', ENDKEY => ''}] 2023-05-10 04:54:03,018 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testRollbackAndDoubleExecution' as deleted. 2023-05-10 04:54:03,018 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1683694443018"}]},"ts":"9223372036854775807"} 2023-05-10 04:54:03,020 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testRollbackAndDoubleExecution state from META 2023-05-10 04:54:03,024 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=52, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-05-10 04:54:03,025 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution in 31 msec 2023-05-10 04:54:03,098 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(1227): Checking to see if procedure is done pid=52 2023-05-10 04:54:03,098 INFO [Listener at localhost/45007] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testRollbackAndDoubleExecution, procId: 52 completed 2023-05-10 04:54:03,112 INFO [Listener at localhost/45007] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testRollbackAndDoubleExecution Thread=395 (was 398), OpenFileDescriptor=845 (was 848), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=308 (was 336), ProcessCount=175 (was 175), AvailableMemoryMB=9650 (was 9724) 2023-05-10 04:54:03,125 INFO [Listener at localhost/45007] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionNoStoreFile Thread=395, OpenFileDescriptor=845, MaxFileDescriptor=60000, SystemLoadAverage=308, ProcessCount=175, AvailableMemoryMB=9649 2023-05-10 04:54:03,126 WARN [Listener at localhost/45007] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-05-10 04:54:03,128 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-05-10 04:54:03,128 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-05-10 04:54:03,129 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(1029): Stored pid=53, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-10 04:54:03,131 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_PRE_OPERATION 2023-05-10 04:54:03,132 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-05-10 04:54:03,133 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionNoStoreFile/a7ae61c05a799883e33cfd9dfad56fa1 2023-05-10 04:54:03,134 DEBUG [HFileArchiver-5] backup.HFileArchiver(153): Directory hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionNoStoreFile/a7ae61c05a799883e33cfd9dfad56fa1 empty. 2023-05-10 04:54:03,134 DEBUG [HFileArchiver-5] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionNoStoreFile/a7ae61c05a799883e33cfd9dfad56fa1 2023-05-10 04:54:03,134 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionNoStoreFile regions 2023-05-10 04:54:03,144 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionNoStoreFile/.tabledesc/.tableinfo.0000000001 2023-05-10 04:54:03,145 INFO [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(7675): creating {ENCODED => a7ae61c05a799883e33cfd9dfad56fa1, NAME => 'testSplitTableRegionNoStoreFile,,1683694443128.a7ae61c05a799883e33cfd9dfad56fa1.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionNoStoreFile', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp 2023-05-10 04:54:03,153 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(866): Instantiated testSplitTableRegionNoStoreFile,,1683694443128.a7ae61c05a799883e33cfd9dfad56fa1.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-10 04:54:03,154 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1604): Closing a7ae61c05a799883e33cfd9dfad56fa1, disabling compactions & flushes 2023-05-10 04:54:03,154 INFO [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1626): Closing region testSplitTableRegionNoStoreFile,,1683694443128.a7ae61c05a799883e33cfd9dfad56fa1. 2023-05-10 04:54:03,154 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,,1683694443128.a7ae61c05a799883e33cfd9dfad56fa1. 2023-05-10 04:54:03,154 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionNoStoreFile,,1683694443128.a7ae61c05a799883e33cfd9dfad56fa1. after waiting 0 ms 2023-05-10 04:54:03,154 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionNoStoreFile,,1683694443128.a7ae61c05a799883e33cfd9dfad56fa1. 2023-05-10 04:54:03,154 INFO [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1838): Closed testSplitTableRegionNoStoreFile,,1683694443128.a7ae61c05a799883e33cfd9dfad56fa1. 2023-05-10 04:54:03,154 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1558): Region close journal for a7ae61c05a799883e33cfd9dfad56fa1: 2023-05-10 04:54:03,156 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_ADD_TO_META 2023-05-10 04:54:03,156 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,,1683694443128.a7ae61c05a799883e33cfd9dfad56fa1.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1683694443156"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683694443156"}]},"ts":"1683694443156"} 2023-05-10 04:54:03,158 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-05-10 04:54:03,159 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-05-10 04:54:03,159 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683694443159"}]},"ts":"1683694443159"} 2023-05-10 04:54:03,160 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=ENABLING in hbase:meta 2023-05-10 04:54:03,163 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-05-10 04:54:03,164 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-05-10 04:54:03,164 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-05-10 04:54:03,164 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-05-10 04:54:03,164 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-05-10 04:54:03,164 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=54, ppid=53, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=a7ae61c05a799883e33cfd9dfad56fa1, ASSIGN}] 2023-05-10 04:54:03,166 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=54, ppid=53, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=a7ae61c05a799883e33cfd9dfad56fa1, ASSIGN 2023-05-10 04:54:03,167 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=54, ppid=53, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=a7ae61c05a799883e33cfd9dfad56fa1, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,36665,1683694419452; forceNewPlan=false, retain=false 2023-05-10 04:54:03,317 INFO [jenkins-hbase4:37903] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-05-10 04:54:03,318 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=54 updating hbase:meta row=a7ae61c05a799883e33cfd9dfad56fa1, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:54:03,319 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1683694443128.a7ae61c05a799883e33cfd9dfad56fa1.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1683694443318"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683694443318"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683694443318"}]},"ts":"1683694443318"} 2023-05-10 04:54:03,320 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=55, ppid=54, state=RUNNABLE; OpenRegionProcedure a7ae61c05a799883e33cfd9dfad56fa1, server=jenkins-hbase4.apache.org,36665,1683694419452}] 2023-05-10 04:54:03,479 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionNoStoreFile,,1683694443128.a7ae61c05a799883e33cfd9dfad56fa1. 2023-05-10 04:54:03,479 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => a7ae61c05a799883e33cfd9dfad56fa1, NAME => 'testSplitTableRegionNoStoreFile,,1683694443128.a7ae61c05a799883e33cfd9dfad56fa1.', STARTKEY => '', ENDKEY => ''} 2023-05-10 04:54:03,479 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-05-10 04:54:03,480 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionNoStoreFile a7ae61c05a799883e33cfd9dfad56fa1 2023-05-10 04:54:03,480 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionNoStoreFile,,1683694443128.a7ae61c05a799883e33cfd9dfad56fa1.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-10 04:54:03,480 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for a7ae61c05a799883e33cfd9dfad56fa1 2023-05-10 04:54:03,480 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for a7ae61c05a799883e33cfd9dfad56fa1 2023-05-10 04:54:03,482 INFO [StoreOpener-a7ae61c05a799883e33cfd9dfad56fa1-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region a7ae61c05a799883e33cfd9dfad56fa1 2023-05-10 04:54:03,484 DEBUG [StoreOpener-a7ae61c05a799883e33cfd9dfad56fa1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionNoStoreFile/a7ae61c05a799883e33cfd9dfad56fa1/cf1 2023-05-10 04:54:03,484 DEBUG [StoreOpener-a7ae61c05a799883e33cfd9dfad56fa1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionNoStoreFile/a7ae61c05a799883e33cfd9dfad56fa1/cf1 2023-05-10 04:54:03,484 INFO [StoreOpener-a7ae61c05a799883e33cfd9dfad56fa1-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a7ae61c05a799883e33cfd9dfad56fa1 columnFamilyName cf1 2023-05-10 04:54:03,485 INFO [StoreOpener-a7ae61c05a799883e33cfd9dfad56fa1-1] regionserver.HStore(310): Store=a7ae61c05a799883e33cfd9dfad56fa1/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-10 04:54:03,485 INFO [StoreOpener-a7ae61c05a799883e33cfd9dfad56fa1-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region a7ae61c05a799883e33cfd9dfad56fa1 2023-05-10 04:54:03,487 DEBUG [StoreOpener-a7ae61c05a799883e33cfd9dfad56fa1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionNoStoreFile/a7ae61c05a799883e33cfd9dfad56fa1/cf2 2023-05-10 04:54:03,487 DEBUG [StoreOpener-a7ae61c05a799883e33cfd9dfad56fa1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionNoStoreFile/a7ae61c05a799883e33cfd9dfad56fa1/cf2 2023-05-10 04:54:03,487 INFO [StoreOpener-a7ae61c05a799883e33cfd9dfad56fa1-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region a7ae61c05a799883e33cfd9dfad56fa1 columnFamilyName cf2 2023-05-10 04:54:03,488 INFO [StoreOpener-a7ae61c05a799883e33cfd9dfad56fa1-1] regionserver.HStore(310): Store=a7ae61c05a799883e33cfd9dfad56fa1/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-10 04:54:03,489 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionNoStoreFile/a7ae61c05a799883e33cfd9dfad56fa1 2023-05-10 04:54:03,489 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionNoStoreFile/a7ae61c05a799883e33cfd9dfad56fa1 2023-05-10 04:54:03,491 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionNoStoreFile descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-05-10 04:54:03,492 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for a7ae61c05a799883e33cfd9dfad56fa1 2023-05-10 04:54:03,494 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionNoStoreFile/a7ae61c05a799883e33cfd9dfad56fa1/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-05-10 04:54:03,494 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened a7ae61c05a799883e33cfd9dfad56fa1; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59485974, jitterRate=-0.11358991265296936}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-05-10 04:54:03,494 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for a7ae61c05a799883e33cfd9dfad56fa1: 2023-05-10 04:54:03,495 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2334): Post open deploy tasks for testSplitTableRegionNoStoreFile,,1683694443128.a7ae61c05a799883e33cfd9dfad56fa1., pid=55, masterSystemTime=1683694443472 2023-05-10 04:54:03,497 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2361): Finished post open deploy task for testSplitTableRegionNoStoreFile,,1683694443128.a7ae61c05a799883e33cfd9dfad56fa1. 2023-05-10 04:54:03,497 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionNoStoreFile,,1683694443128.a7ae61c05a799883e33cfd9dfad56fa1. 2023-05-10 04:54:03,497 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=54 updating hbase:meta row=a7ae61c05a799883e33cfd9dfad56fa1, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:54:03,497 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionNoStoreFile,,1683694443128.a7ae61c05a799883e33cfd9dfad56fa1.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1683694443497"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1683694443497"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1683694443497"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683694443497"}]},"ts":"1683694443497"} 2023-05-10 04:54:03,501 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=55, resume processing ppid=54 2023-05-10 04:54:03,501 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=55, ppid=54, state=SUCCESS; OpenRegionProcedure a7ae61c05a799883e33cfd9dfad56fa1, server=jenkins-hbase4.apache.org,36665,1683694419452 in 179 msec 2023-05-10 04:54:03,504 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=54, resume processing ppid=53 2023-05-10 04:54:03,504 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=54, ppid=53, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=a7ae61c05a799883e33cfd9dfad56fa1, ASSIGN in 337 msec 2023-05-10 04:54:03,505 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-05-10 04:54:03,505 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683694443505"}]},"ts":"1683694443505"} 2023-05-10 04:54:03,506 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=ENABLED in hbase:meta 2023-05-10 04:54:03,510 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_POST_OPERATION 2023-05-10 04:54:03,511 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile in 382 msec 2023-05-10 04:54:03,630 DEBUG [Listener at localhost/45007] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:54:03,632 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(1029): Stored pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=a7ae61c05a799883e33cfd9dfad56fa1, daughterA=944df92d059326a46982ff7979f8a940, daughterB=fa1eb74e4c5521990fb9118cc6dfd407 2023-05-10 04:54:03,633 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=a7ae61c05a799883e33cfd9dfad56fa1, daughterA=944df92d059326a46982ff7979f8a940, daughterB=fa1eb74e4c5521990fb9118cc6dfd407 2023-05-10 04:54:03,633 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=a7ae61c05a799883e33cfd9dfad56fa1, daughterA=944df92d059326a46982ff7979f8a940, daughterB=fa1eb74e4c5521990fb9118cc6dfd407 2023-05-10 04:54:03,633 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=a7ae61c05a799883e33cfd9dfad56fa1, daughterA=944df92d059326a46982ff7979f8a940, daughterB=fa1eb74e4c5521990fb9118cc6dfd407 2023-05-10 04:54:03,635 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=57, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=a7ae61c05a799883e33cfd9dfad56fa1, UNASSIGN}] 2023-05-10 04:54:03,635 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=57, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=a7ae61c05a799883e33cfd9dfad56fa1, UNASSIGN 2023-05-10 04:54:03,636 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=57 updating hbase:meta row=a7ae61c05a799883e33cfd9dfad56fa1, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:54:03,636 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1683694443128.a7ae61c05a799883e33cfd9dfad56fa1.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1683694443636"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683694443636"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683694443636"}]},"ts":"1683694443636"} 2023-05-10 04:54:03,638 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=58, ppid=57, state=RUNNABLE; CloseRegionProcedure a7ae61c05a799883e33cfd9dfad56fa1, server=jenkins-hbase4.apache.org,36665,1683694419452}] 2023-05-10 04:54:03,790 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close a7ae61c05a799883e33cfd9dfad56fa1 2023-05-10 04:54:03,791 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing a7ae61c05a799883e33cfd9dfad56fa1, disabling compactions & flushes 2023-05-10 04:54:03,791 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionNoStoreFile,,1683694443128.a7ae61c05a799883e33cfd9dfad56fa1. 2023-05-10 04:54:03,791 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,,1683694443128.a7ae61c05a799883e33cfd9dfad56fa1. 2023-05-10 04:54:03,791 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionNoStoreFile,,1683694443128.a7ae61c05a799883e33cfd9dfad56fa1. after waiting 0 ms 2023-05-10 04:54:03,791 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionNoStoreFile,,1683694443128.a7ae61c05a799883e33cfd9dfad56fa1. 2023-05-10 04:54:03,795 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionNoStoreFile/a7ae61c05a799883e33cfd9dfad56fa1/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-05-10 04:54:03,795 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-10 04:54:03,796 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionNoStoreFile,,1683694443128.a7ae61c05a799883e33cfd9dfad56fa1. 2023-05-10 04:54:03,796 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for a7ae61c05a799883e33cfd9dfad56fa1: 2023-05-10 04:54:03,797 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed a7ae61c05a799883e33cfd9dfad56fa1 2023-05-10 04:54:03,798 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=57 updating hbase:meta row=a7ae61c05a799883e33cfd9dfad56fa1, regionState=CLOSED 2023-05-10 04:54:03,798 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,,1683694443128.a7ae61c05a799883e33cfd9dfad56fa1.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1683694443798"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683694443798"}]},"ts":"1683694443798"} 2023-05-10 04:54:03,800 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=58, resume processing ppid=57 2023-05-10 04:54:03,800 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=58, ppid=57, state=SUCCESS; CloseRegionProcedure a7ae61c05a799883e33cfd9dfad56fa1, server=jenkins-hbase4.apache.org,36665,1683694419452 in 161 msec 2023-05-10 04:54:03,802 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=57, resume processing ppid=56 2023-05-10 04:54:03,802 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=57, ppid=56, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=a7ae61c05a799883e33cfd9dfad56fa1, UNASSIGN in 165 msec 2023-05-10 04:54:03,826 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionNoStoreFile/944df92d059326a46982ff7979f8a940/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=-1 2023-05-10 04:54:03,827 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionNoStoreFile/fa1eb74e4c5521990fb9118cc6dfd407/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=-1 2023-05-10 04:54:03,829 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1683694443128.a7ae61c05a799883e33cfd9dfad56fa1.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1683694443829"},{"qualifier":"splitA","vlen":67,"tag":[],"timestamp":"1683694443829"},{"qualifier":"splitB","vlen":67,"tag":[],"timestamp":"1683694443829"}]},"ts":"1683694443829"} 2023-05-10 04:54:03,829 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1683694443631.944df92d059326a46982ff7979f8a940.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1683694443829"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683694443829"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683694443829"}]},"ts":"1683694443829"} 2023-05-10 04:54:03,829 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,41,1683694443631.fa1eb74e4c5521990fb9118cc6dfd407.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1683694443829"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683694443829"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683694443829"}]},"ts":"1683694443829"} 2023-05-10 04:54:03,832 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=59, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=944df92d059326a46982ff7979f8a940, ASSIGN}, {pid=60, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=fa1eb74e4c5521990fb9118cc6dfd407, ASSIGN}] 2023-05-10 04:54:03,833 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=60, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=fa1eb74e4c5521990fb9118cc6dfd407, ASSIGN 2023-05-10 04:54:03,833 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=60, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=fa1eb74e4c5521990fb9118cc6dfd407, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,36665,1683694419452; forceNewPlan=false, retain=false 2023-05-10 04:54:03,834 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=59, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=944df92d059326a46982ff7979f8a940, ASSIGN 2023-05-10 04:54:03,834 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=59, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=944df92d059326a46982ff7979f8a940, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,36665,1683694419452; forceNewPlan=false, retain=false 2023-05-10 04:54:03,983 INFO [jenkins-hbase4:37903] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-05-10 04:54:03,985 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=59 updating hbase:meta row=944df92d059326a46982ff7979f8a940, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:54:03,985 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1683694443631.944df92d059326a46982ff7979f8a940.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1683694443985"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683694443985"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683694443985"}]},"ts":"1683694443985"} 2023-05-10 04:54:03,987 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=61, ppid=59, state=RUNNABLE; OpenRegionProcedure 944df92d059326a46982ff7979f8a940, server=jenkins-hbase4.apache.org,36665,1683694419452}] 2023-05-10 04:54:03,990 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=60 updating hbase:meta row=fa1eb74e4c5521990fb9118cc6dfd407, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:54:03,990 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,41,1683694443631.fa1eb74e4c5521990fb9118cc6dfd407.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1683694443990"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683694443990"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683694443990"}]},"ts":"1683694443990"} 2023-05-10 04:54:03,991 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=62, ppid=60, state=RUNNABLE; OpenRegionProcedure fa1eb74e4c5521990fb9118cc6dfd407, server=jenkins-hbase4.apache.org,36665,1683694419452}] 2023-05-10 04:54:04,142 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionNoStoreFile,41,1683694443631.fa1eb74e4c5521990fb9118cc6dfd407. 2023-05-10 04:54:04,142 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => fa1eb74e4c5521990fb9118cc6dfd407, NAME => 'testSplitTableRegionNoStoreFile,41,1683694443631.fa1eb74e4c5521990fb9118cc6dfd407.', STARTKEY => '41', ENDKEY => ''} 2023-05-10 04:54:04,142 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-05-10 04:54:04,143 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionNoStoreFile fa1eb74e4c5521990fb9118cc6dfd407 2023-05-10 04:54:04,143 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionNoStoreFile,41,1683694443631.fa1eb74e4c5521990fb9118cc6dfd407.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-10 04:54:04,143 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for fa1eb74e4c5521990fb9118cc6dfd407 2023-05-10 04:54:04,143 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for fa1eb74e4c5521990fb9118cc6dfd407 2023-05-10 04:54:04,144 INFO [StoreOpener-fa1eb74e4c5521990fb9118cc6dfd407-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region fa1eb74e4c5521990fb9118cc6dfd407 2023-05-10 04:54:04,145 DEBUG [StoreOpener-fa1eb74e4c5521990fb9118cc6dfd407-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionNoStoreFile/fa1eb74e4c5521990fb9118cc6dfd407/cf1 2023-05-10 04:54:04,145 DEBUG [StoreOpener-fa1eb74e4c5521990fb9118cc6dfd407-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionNoStoreFile/fa1eb74e4c5521990fb9118cc6dfd407/cf1 2023-05-10 04:54:04,146 INFO [StoreOpener-fa1eb74e4c5521990fb9118cc6dfd407-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region fa1eb74e4c5521990fb9118cc6dfd407 columnFamilyName cf1 2023-05-10 04:54:04,146 INFO [StoreOpener-fa1eb74e4c5521990fb9118cc6dfd407-1] regionserver.HStore(310): Store=fa1eb74e4c5521990fb9118cc6dfd407/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-10 04:54:04,146 INFO [StoreOpener-fa1eb74e4c5521990fb9118cc6dfd407-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region fa1eb74e4c5521990fb9118cc6dfd407 2023-05-10 04:54:04,147 DEBUG [StoreOpener-fa1eb74e4c5521990fb9118cc6dfd407-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionNoStoreFile/fa1eb74e4c5521990fb9118cc6dfd407/cf2 2023-05-10 04:54:04,147 DEBUG [StoreOpener-fa1eb74e4c5521990fb9118cc6dfd407-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionNoStoreFile/fa1eb74e4c5521990fb9118cc6dfd407/cf2 2023-05-10 04:54:04,148 INFO [StoreOpener-fa1eb74e4c5521990fb9118cc6dfd407-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region fa1eb74e4c5521990fb9118cc6dfd407 columnFamilyName cf2 2023-05-10 04:54:04,148 INFO [StoreOpener-fa1eb74e4c5521990fb9118cc6dfd407-1] regionserver.HStore(310): Store=fa1eb74e4c5521990fb9118cc6dfd407/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-10 04:54:04,149 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionNoStoreFile/fa1eb74e4c5521990fb9118cc6dfd407 2023-05-10 04:54:04,150 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionNoStoreFile/fa1eb74e4c5521990fb9118cc6dfd407 2023-05-10 04:54:04,151 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionNoStoreFile descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-05-10 04:54:04,152 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for fa1eb74e4c5521990fb9118cc6dfd407 2023-05-10 04:54:04,153 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened fa1eb74e4c5521990fb9118cc6dfd407; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71419920, jitterRate=0.0642397403717041}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-05-10 04:54:04,153 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for fa1eb74e4c5521990fb9118cc6dfd407: 2023-05-10 04:54:04,154 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2334): Post open deploy tasks for testSplitTableRegionNoStoreFile,41,1683694443631.fa1eb74e4c5521990fb9118cc6dfd407., pid=62, masterSystemTime=1683694444139 2023-05-10 04:54:04,155 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2361): Finished post open deploy task for testSplitTableRegionNoStoreFile,41,1683694443631.fa1eb74e4c5521990fb9118cc6dfd407. 2023-05-10 04:54:04,155 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionNoStoreFile,41,1683694443631.fa1eb74e4c5521990fb9118cc6dfd407. 2023-05-10 04:54:04,155 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionNoStoreFile,,1683694443631.944df92d059326a46982ff7979f8a940. 2023-05-10 04:54:04,156 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 944df92d059326a46982ff7979f8a940, NAME => 'testSplitTableRegionNoStoreFile,,1683694443631.944df92d059326a46982ff7979f8a940.', STARTKEY => '', ENDKEY => '41'} 2023-05-10 04:54:04,156 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=60 updating hbase:meta row=fa1eb74e4c5521990fb9118cc6dfd407, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:54:04,156 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionNoStoreFile,41,1683694443631.fa1eb74e4c5521990fb9118cc6dfd407.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1683694444156"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1683694444156"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1683694444156"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683694444156"}]},"ts":"1683694444156"} 2023-05-10 04:54:04,156 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-05-10 04:54:04,156 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionNoStoreFile 944df92d059326a46982ff7979f8a940 2023-05-10 04:54:04,156 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionNoStoreFile,,1683694443631.944df92d059326a46982ff7979f8a940.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-10 04:54:04,156 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 944df92d059326a46982ff7979f8a940 2023-05-10 04:54:04,156 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 944df92d059326a46982ff7979f8a940 2023-05-10 04:54:04,158 INFO [StoreOpener-944df92d059326a46982ff7979f8a940-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 944df92d059326a46982ff7979f8a940 2023-05-10 04:54:04,159 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=62, resume processing ppid=60 2023-05-10 04:54:04,159 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=62, ppid=60, state=SUCCESS; OpenRegionProcedure fa1eb74e4c5521990fb9118cc6dfd407, server=jenkins-hbase4.apache.org,36665,1683694419452 in 166 msec 2023-05-10 04:54:04,159 DEBUG [StoreOpener-944df92d059326a46982ff7979f8a940-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionNoStoreFile/944df92d059326a46982ff7979f8a940/cf1 2023-05-10 04:54:04,159 DEBUG [StoreOpener-944df92d059326a46982ff7979f8a940-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionNoStoreFile/944df92d059326a46982ff7979f8a940/cf1 2023-05-10 04:54:04,160 INFO [StoreOpener-944df92d059326a46982ff7979f8a940-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 944df92d059326a46982ff7979f8a940 columnFamilyName cf1 2023-05-10 04:54:04,160 INFO [StoreOpener-944df92d059326a46982ff7979f8a940-1] regionserver.HStore(310): Store=944df92d059326a46982ff7979f8a940/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-10 04:54:04,160 INFO [StoreOpener-944df92d059326a46982ff7979f8a940-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 944df92d059326a46982ff7979f8a940 2023-05-10 04:54:04,160 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=60, ppid=56, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=fa1eb74e4c5521990fb9118cc6dfd407, ASSIGN in 327 msec 2023-05-10 04:54:04,161 DEBUG [StoreOpener-944df92d059326a46982ff7979f8a940-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionNoStoreFile/944df92d059326a46982ff7979f8a940/cf2 2023-05-10 04:54:04,161 DEBUG [StoreOpener-944df92d059326a46982ff7979f8a940-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionNoStoreFile/944df92d059326a46982ff7979f8a940/cf2 2023-05-10 04:54:04,162 INFO [StoreOpener-944df92d059326a46982ff7979f8a940-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 944df92d059326a46982ff7979f8a940 columnFamilyName cf2 2023-05-10 04:54:04,162 INFO [StoreOpener-944df92d059326a46982ff7979f8a940-1] regionserver.HStore(310): Store=944df92d059326a46982ff7979f8a940/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-10 04:54:04,163 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionNoStoreFile/944df92d059326a46982ff7979f8a940 2023-05-10 04:54:04,164 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionNoStoreFile/944df92d059326a46982ff7979f8a940 2023-05-10 04:54:04,165 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionNoStoreFile descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-05-10 04:54:04,167 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 944df92d059326a46982ff7979f8a940 2023-05-10 04:54:04,167 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 944df92d059326a46982ff7979f8a940; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72437870, jitterRate=0.07940837740898132}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-05-10 04:54:04,167 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 944df92d059326a46982ff7979f8a940: 2023-05-10 04:54:04,168 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2334): Post open deploy tasks for testSplitTableRegionNoStoreFile,,1683694443631.944df92d059326a46982ff7979f8a940., pid=61, masterSystemTime=1683694444139 2023-05-10 04:54:04,169 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2361): Finished post open deploy task for testSplitTableRegionNoStoreFile,,1683694443631.944df92d059326a46982ff7979f8a940. 2023-05-10 04:54:04,170 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionNoStoreFile,,1683694443631.944df92d059326a46982ff7979f8a940. 2023-05-10 04:54:04,170 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=59 updating hbase:meta row=944df92d059326a46982ff7979f8a940, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:54:04,170 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionNoStoreFile,,1683694443631.944df92d059326a46982ff7979f8a940.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1683694444170"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1683694444170"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1683694444170"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683694444170"}]},"ts":"1683694444170"} 2023-05-10 04:54:04,173 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=61, resume processing ppid=59 2023-05-10 04:54:04,173 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=61, ppid=59, state=SUCCESS; OpenRegionProcedure 944df92d059326a46982ff7979f8a940, server=jenkins-hbase4.apache.org,36665,1683694419452 in 184 msec 2023-05-10 04:54:04,174 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=59, resume processing ppid=56 2023-05-10 04:54:04,174 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=59, ppid=56, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=944df92d059326a46982ff7979f8a940, ASSIGN in 341 msec 2023-05-10 04:54:04,175 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=a7ae61c05a799883e33cfd9dfad56fa1, daughterA=944df92d059326a46982ff7979f8a940, daughterB=fa1eb74e4c5521990fb9118cc6dfd407 in 543 msec 2023-05-10 04:54:04,385 DEBUG [hconnection-0x248fdd40-shared-pool-11] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-05-10 04:54:04,388 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:54092, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-05-10 04:54:04,389 DEBUG [Listener at localhost/45007] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-05-10 04:54:04,392 WARN [Listener at localhost/45007] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-05-10 04:54:04,393 INFO [Listener at localhost/45007] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionNoStoreFile 2023-05-10 04:54:04,393 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testSplitTableRegionNoStoreFile 2023-05-10 04:54:04,394 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] procedure2.ProcedureExecutor(1029): Stored pid=63, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-10 04:54:04,396 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(1227): Checking to see if procedure is done pid=63 2023-05-10 04:54:04,397 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683694444397"}]},"ts":"1683694444397"} 2023-05-10 04:54:04,398 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=DISABLING in hbase:meta 2023-05-10 04:54:04,399 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionNoStoreFile to state=DISABLING 2023-05-10 04:54:04,400 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=64, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=944df92d059326a46982ff7979f8a940, UNASSIGN}, {pid=65, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=fa1eb74e4c5521990fb9118cc6dfd407, UNASSIGN}] 2023-05-10 04:54:04,401 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=65, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=fa1eb74e4c5521990fb9118cc6dfd407, UNASSIGN 2023-05-10 04:54:04,401 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=65 updating hbase:meta row=fa1eb74e4c5521990fb9118cc6dfd407, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:54:04,401 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,41,1683694443631.fa1eb74e4c5521990fb9118cc6dfd407.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1683694444401"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683694444401"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683694444401"}]},"ts":"1683694444401"} 2023-05-10 04:54:04,403 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=66, ppid=65, state=RUNNABLE; CloseRegionProcedure fa1eb74e4c5521990fb9118cc6dfd407, server=jenkins-hbase4.apache.org,36665,1683694419452}] 2023-05-10 04:54:04,405 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=64, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=944df92d059326a46982ff7979f8a940, UNASSIGN 2023-05-10 04:54:04,405 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=64 updating hbase:meta row=944df92d059326a46982ff7979f8a940, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:54:04,405 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1683694443631.944df92d059326a46982ff7979f8a940.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1683694444405"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683694444405"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683694444405"}]},"ts":"1683694444405"} 2023-05-10 04:54:04,407 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=67, ppid=64, state=RUNNABLE; CloseRegionProcedure 944df92d059326a46982ff7979f8a940, server=jenkins-hbase4.apache.org,36665,1683694419452}] 2023-05-10 04:54:04,497 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(1227): Checking to see if procedure is done pid=63 2023-05-10 04:54:04,554 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close fa1eb74e4c5521990fb9118cc6dfd407 2023-05-10 04:54:04,555 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing fa1eb74e4c5521990fb9118cc6dfd407, disabling compactions & flushes 2023-05-10 04:54:04,556 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionNoStoreFile,41,1683694443631.fa1eb74e4c5521990fb9118cc6dfd407. 2023-05-10 04:54:04,556 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,41,1683694443631.fa1eb74e4c5521990fb9118cc6dfd407. 2023-05-10 04:54:04,556 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionNoStoreFile,41,1683694443631.fa1eb74e4c5521990fb9118cc6dfd407. after waiting 0 ms 2023-05-10 04:54:04,556 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionNoStoreFile,41,1683694443631.fa1eb74e4c5521990fb9118cc6dfd407. 2023-05-10 04:54:04,559 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionNoStoreFile/fa1eb74e4c5521990fb9118cc6dfd407/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-05-10 04:54:04,560 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-10 04:54:04,560 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionNoStoreFile,41,1683694443631.fa1eb74e4c5521990fb9118cc6dfd407. 2023-05-10 04:54:04,560 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for fa1eb74e4c5521990fb9118cc6dfd407: 2023-05-10 04:54:04,562 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed fa1eb74e4c5521990fb9118cc6dfd407 2023-05-10 04:54:04,562 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 944df92d059326a46982ff7979f8a940 2023-05-10 04:54:04,563 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 944df92d059326a46982ff7979f8a940, disabling compactions & flushes 2023-05-10 04:54:04,563 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionNoStoreFile,,1683694443631.944df92d059326a46982ff7979f8a940. 2023-05-10 04:54:04,563 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,,1683694443631.944df92d059326a46982ff7979f8a940. 2023-05-10 04:54:04,563 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionNoStoreFile,,1683694443631.944df92d059326a46982ff7979f8a940. after waiting 0 ms 2023-05-10 04:54:04,563 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionNoStoreFile,,1683694443631.944df92d059326a46982ff7979f8a940. 2023-05-10 04:54:04,563 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=65 updating hbase:meta row=fa1eb74e4c5521990fb9118cc6dfd407, regionState=CLOSED 2023-05-10 04:54:04,563 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,41,1683694443631.fa1eb74e4c5521990fb9118cc6dfd407.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1683694444563"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683694444563"}]},"ts":"1683694444563"} 2023-05-10 04:54:04,566 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=66, resume processing ppid=65 2023-05-10 04:54:04,566 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=66, ppid=65, state=SUCCESS; CloseRegionProcedure fa1eb74e4c5521990fb9118cc6dfd407, server=jenkins-hbase4.apache.org,36665,1683694419452 in 162 msec 2023-05-10 04:54:04,567 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionNoStoreFile/944df92d059326a46982ff7979f8a940/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-05-10 04:54:04,567 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-10 04:54:04,567 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionNoStoreFile,,1683694443631.944df92d059326a46982ff7979f8a940. 2023-05-10 04:54:04,567 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 944df92d059326a46982ff7979f8a940: 2023-05-10 04:54:04,567 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=65, ppid=63, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=fa1eb74e4c5521990fb9118cc6dfd407, UNASSIGN in 166 msec 2023-05-10 04:54:04,568 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 944df92d059326a46982ff7979f8a940 2023-05-10 04:54:04,569 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=64 updating hbase:meta row=944df92d059326a46982ff7979f8a940, regionState=CLOSED 2023-05-10 04:54:04,569 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,,1683694443631.944df92d059326a46982ff7979f8a940.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1683694444569"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683694444569"}]},"ts":"1683694444569"} 2023-05-10 04:54:04,571 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=67, resume processing ppid=64 2023-05-10 04:54:04,571 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=67, ppid=64, state=SUCCESS; CloseRegionProcedure 944df92d059326a46982ff7979f8a940, server=jenkins-hbase4.apache.org,36665,1683694419452 in 163 msec 2023-05-10 04:54:04,573 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=64, resume processing ppid=63 2023-05-10 04:54:04,573 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=64, ppid=63, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=944df92d059326a46982ff7979f8a940, UNASSIGN in 171 msec 2023-05-10 04:54:04,573 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683694444573"}]},"ts":"1683694444573"} 2023-05-10 04:54:04,574 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=DISABLED in hbase:meta 2023-05-10 04:54:04,579 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionNoStoreFile to state=DISABLED 2023-05-10 04:54:04,581 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile in 186 msec 2023-05-10 04:54:04,699 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(1227): Checking to see if procedure is done pid=63 2023-05-10 04:54:04,699 INFO [Listener at localhost/45007] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionNoStoreFile, procId: 63 completed 2023-05-10 04:54:04,700 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete testSplitTableRegionNoStoreFile 2023-05-10 04:54:04,700 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] procedure2.ProcedureExecutor(1029): Stored pid=68, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-10 04:54:04,702 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=68, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-10 04:54:04,703 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=68, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-10 04:54:04,703 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(1227): Checking to see if procedure is done pid=68 2023-05-10 04:54:04,709 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionNoStoreFile/a7ae61c05a799883e33cfd9dfad56fa1 2023-05-10 04:54:04,709 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionNoStoreFile/fa1eb74e4c5521990fb9118cc6dfd407 2023-05-10 04:54:04,709 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionNoStoreFile/944df92d059326a46982ff7979f8a940 2023-05-10 04:54:04,711 DEBUG [HFileArchiver-6] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionNoStoreFile/944df92d059326a46982ff7979f8a940/cf1, FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionNoStoreFile/944df92d059326a46982ff7979f8a940/cf2, FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionNoStoreFile/944df92d059326a46982ff7979f8a940/recovered.edits] 2023-05-10 04:54:04,711 DEBUG [HFileArchiver-7] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionNoStoreFile/fa1eb74e4c5521990fb9118cc6dfd407/cf1, FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionNoStoreFile/fa1eb74e4c5521990fb9118cc6dfd407/cf2, FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionNoStoreFile/fa1eb74e4c5521990fb9118cc6dfd407/recovered.edits] 2023-05-10 04:54:04,712 DEBUG [HFileArchiver-3] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionNoStoreFile/a7ae61c05a799883e33cfd9dfad56fa1/cf1, FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionNoStoreFile/a7ae61c05a799883e33cfd9dfad56fa1/cf2, FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionNoStoreFile/a7ae61c05a799883e33cfd9dfad56fa1/recovered.edits] 2023-05-10 04:54:04,718 DEBUG [HFileArchiver-6] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionNoStoreFile/944df92d059326a46982ff7979f8a940/recovered.edits/7.seqid to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionNoStoreFile/944df92d059326a46982ff7979f8a940/recovered.edits/7.seqid 2023-05-10 04:54:04,718 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionNoStoreFile/a7ae61c05a799883e33cfd9dfad56fa1/recovered.edits/4.seqid to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionNoStoreFile/a7ae61c05a799883e33cfd9dfad56fa1/recovered.edits/4.seqid 2023-05-10 04:54:04,718 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionNoStoreFile/fa1eb74e4c5521990fb9118cc6dfd407/recovered.edits/7.seqid to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionNoStoreFile/fa1eb74e4c5521990fb9118cc6dfd407/recovered.edits/7.seqid 2023-05-10 04:54:04,719 DEBUG [HFileArchiver-6] backup.HFileArchiver(596): Deleted hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionNoStoreFile/944df92d059326a46982ff7979f8a940 2023-05-10 04:54:04,719 DEBUG [HFileArchiver-7] backup.HFileArchiver(596): Deleted hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionNoStoreFile/fa1eb74e4c5521990fb9118cc6dfd407 2023-05-10 04:54:04,719 DEBUG [HFileArchiver-3] backup.HFileArchiver(596): Deleted hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionNoStoreFile/a7ae61c05a799883e33cfd9dfad56fa1 2023-05-10 04:54:04,719 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionNoStoreFile regions 2023-05-10 04:54:04,721 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=68, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-10 04:54:04,723 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionNoStoreFile from hbase:meta 2023-05-10 04:54:04,725 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionNoStoreFile' descriptor. 2023-05-10 04:54:04,726 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=68, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-10 04:54:04,726 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionNoStoreFile' from region states. 2023-05-10 04:54:04,726 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile,,1683694443128.a7ae61c05a799883e33cfd9dfad56fa1.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1683694444726"}]},"ts":"9223372036854775807"} 2023-05-10 04:54:04,726 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile,,1683694443631.944df92d059326a46982ff7979f8a940.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1683694444726"}]},"ts":"9223372036854775807"} 2023-05-10 04:54:04,726 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile,41,1683694443631.fa1eb74e4c5521990fb9118cc6dfd407.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1683694444726"}]},"ts":"9223372036854775807"} 2023-05-10 04:54:04,728 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-05-10 04:54:04,728 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => a7ae61c05a799883e33cfd9dfad56fa1, NAME => 'testSplitTableRegionNoStoreFile,,1683694443128.a7ae61c05a799883e33cfd9dfad56fa1.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 944df92d059326a46982ff7979f8a940, NAME => 'testSplitTableRegionNoStoreFile,,1683694443631.944df92d059326a46982ff7979f8a940.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => fa1eb74e4c5521990fb9118cc6dfd407, NAME => 'testSplitTableRegionNoStoreFile,41,1683694443631.fa1eb74e4c5521990fb9118cc6dfd407.', STARTKEY => '41', ENDKEY => ''}] 2023-05-10 04:54:04,728 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionNoStoreFile' as deleted. 2023-05-10 04:54:04,728 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1683694444728"}]},"ts":"9223372036854775807"} 2023-05-10 04:54:04,729 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionNoStoreFile state from META 2023-05-10 04:54:04,732 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=68, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-10 04:54:04,734 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile in 32 msec 2023-05-10 04:54:04,804 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(1227): Checking to see if procedure is done pid=68 2023-05-10 04:54:04,804 INFO [Listener at localhost/45007] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionNoStoreFile, procId: 68 completed 2023-05-10 04:54:04,819 INFO [Listener at localhost/45007] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionNoStoreFile Thread=403 (was 395) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-701639159_15 at /127.0.0.1:47324 [Waiting for operation #31] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x248fdd40-metaLookup-shared--pool-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x248fdd40-shared-pool-11 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x248fdd40-shared-pool-12 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-4 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x248fdd40-metaLookup-shared--pool-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-701639159_15 at /127.0.0.1:46868 [Waiting for operation #20] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=849 (was 845) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=308 (was 308), ProcessCount=175 (was 175), AvailableMemoryMB=9648 (was 9649) 2023-05-10 04:54:04,832 INFO [Listener at localhost/45007] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionDeletedRowsDaughter Thread=403, OpenFileDescriptor=849, MaxFileDescriptor=60000, SystemLoadAverage=308, ProcessCount=175, AvailableMemoryMB=9648 2023-05-10 04:54:04,832 WARN [Listener at localhost/45007] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-05-10 04:54:04,834 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-05-10 04:54:04,835 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-05-10 04:54:04,836 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(1029): Stored pid=69, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:04,838 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_PRE_OPERATION 2023-05-10 04:54:04,839 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-05-10 04:54:04,840 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c 2023-05-10 04:54:04,841 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c empty. 2023-05-10 04:54:04,842 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c 2023-05-10 04:54:04,842 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionDeletedRowsDaughter regions 2023-05-10 04:54:04,857 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/.tabledesc/.tableinfo.0000000001 2023-05-10 04:54:04,859 INFO [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(7675): creating {ENCODED => e0a753dfc7b1edf68920ae1543edc38c, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1683694444835.e0a753dfc7b1edf68920ae1543edc38c.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionDeletedRowsDaughter', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp 2023-05-10 04:54:04,871 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(866): Instantiated testSplitTableRegionDeletedRowsDaughter,,1683694444835.e0a753dfc7b1edf68920ae1543edc38c.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-10 04:54:04,872 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1604): Closing e0a753dfc7b1edf68920ae1543edc38c, disabling compactions & flushes 2023-05-10 04:54:04,872 INFO [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1626): Closing region testSplitTableRegionDeletedRowsDaughter,,1683694444835.e0a753dfc7b1edf68920ae1543edc38c. 2023-05-10 04:54:04,872 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,,1683694444835.e0a753dfc7b1edf68920ae1543edc38c. 2023-05-10 04:54:04,872 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,,1683694444835.e0a753dfc7b1edf68920ae1543edc38c. after waiting 0 ms 2023-05-10 04:54:04,872 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,,1683694444835.e0a753dfc7b1edf68920ae1543edc38c. 2023-05-10 04:54:04,872 INFO [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1838): Closed testSplitTableRegionDeletedRowsDaughter,,1683694444835.e0a753dfc7b1edf68920ae1543edc38c. 2023-05-10 04:54:04,872 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1558): Region close journal for e0a753dfc7b1edf68920ae1543edc38c: 2023-05-10 04:54:04,874 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_ADD_TO_META 2023-05-10 04:54:04,874 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,,1683694444835.e0a753dfc7b1edf68920ae1543edc38c.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1683694444874"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683694444874"}]},"ts":"1683694444874"} 2023-05-10 04:54:04,876 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-05-10 04:54:04,877 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-05-10 04:54:04,877 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683694444877"}]},"ts":"1683694444877"} 2023-05-10 04:54:04,878 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=ENABLING in hbase:meta 2023-05-10 04:54:04,880 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-05-10 04:54:04,881 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-05-10 04:54:04,881 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-05-10 04:54:04,881 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-05-10 04:54:04,881 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-05-10 04:54:04,882 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=70, ppid=69, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=e0a753dfc7b1edf68920ae1543edc38c, ASSIGN}] 2023-05-10 04:54:04,883 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=70, ppid=69, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=e0a753dfc7b1edf68920ae1543edc38c, ASSIGN 2023-05-10 04:54:04,884 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=70, ppid=69, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=e0a753dfc7b1edf68920ae1543edc38c, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,37995,1683694419499; forceNewPlan=false, retain=false 2023-05-10 04:54:05,034 INFO [jenkins-hbase4:37903] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-05-10 04:54:05,035 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=70 updating hbase:meta row=e0a753dfc7b1edf68920ae1543edc38c, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:05,035 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1683694444835.e0a753dfc7b1edf68920ae1543edc38c.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1683694445035"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683694445035"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683694445035"}]},"ts":"1683694445035"} 2023-05-10 04:54:05,037 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=71, ppid=70, state=RUNNABLE; OpenRegionProcedure e0a753dfc7b1edf68920ae1543edc38c, server=jenkins-hbase4.apache.org,37995,1683694419499}] 2023-05-10 04:54:05,195 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionDeletedRowsDaughter,,1683694444835.e0a753dfc7b1edf68920ae1543edc38c. 2023-05-10 04:54:05,195 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => e0a753dfc7b1edf68920ae1543edc38c, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1683694444835.e0a753dfc7b1edf68920ae1543edc38c.', STARTKEY => '', ENDKEY => ''} 2023-05-10 04:54:05,195 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-05-10 04:54:05,196 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionDeletedRowsDaughter e0a753dfc7b1edf68920ae1543edc38c 2023-05-10 04:54:05,196 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionDeletedRowsDaughter,,1683694444835.e0a753dfc7b1edf68920ae1543edc38c.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-10 04:54:05,196 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for e0a753dfc7b1edf68920ae1543edc38c 2023-05-10 04:54:05,196 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for e0a753dfc7b1edf68920ae1543edc38c 2023-05-10 04:54:05,197 INFO [StoreOpener-e0a753dfc7b1edf68920ae1543edc38c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region e0a753dfc7b1edf68920ae1543edc38c 2023-05-10 04:54:05,198 DEBUG [StoreOpener-e0a753dfc7b1edf68920ae1543edc38c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf1 2023-05-10 04:54:05,198 DEBUG [StoreOpener-e0a753dfc7b1edf68920ae1543edc38c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf1 2023-05-10 04:54:05,198 INFO [StoreOpener-e0a753dfc7b1edf68920ae1543edc38c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e0a753dfc7b1edf68920ae1543edc38c columnFamilyName cf1 2023-05-10 04:54:05,199 INFO [StoreOpener-e0a753dfc7b1edf68920ae1543edc38c-1] regionserver.HStore(310): Store=e0a753dfc7b1edf68920ae1543edc38c/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-10 04:54:05,199 INFO [StoreOpener-e0a753dfc7b1edf68920ae1543edc38c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region e0a753dfc7b1edf68920ae1543edc38c 2023-05-10 04:54:05,201 DEBUG [StoreOpener-e0a753dfc7b1edf68920ae1543edc38c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf2 2023-05-10 04:54:05,201 DEBUG [StoreOpener-e0a753dfc7b1edf68920ae1543edc38c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf2 2023-05-10 04:54:05,201 INFO [StoreOpener-e0a753dfc7b1edf68920ae1543edc38c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e0a753dfc7b1edf68920ae1543edc38c columnFamilyName cf2 2023-05-10 04:54:05,202 INFO [StoreOpener-e0a753dfc7b1edf68920ae1543edc38c-1] regionserver.HStore(310): Store=e0a753dfc7b1edf68920ae1543edc38c/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-10 04:54:05,203 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c 2023-05-10 04:54:05,203 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c 2023-05-10 04:54:05,206 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionDeletedRowsDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-05-10 04:54:05,207 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for e0a753dfc7b1edf68920ae1543edc38c 2023-05-10 04:54:05,210 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-05-10 04:54:05,210 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened e0a753dfc7b1edf68920ae1543edc38c; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62155243, jitterRate=-0.07381470501422882}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-05-10 04:54:05,210 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for e0a753dfc7b1edf68920ae1543edc38c: 2023-05-10 04:54:05,211 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2334): Post open deploy tasks for testSplitTableRegionDeletedRowsDaughter,,1683694444835.e0a753dfc7b1edf68920ae1543edc38c., pid=71, masterSystemTime=1683694445189 2023-05-10 04:54:05,213 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2361): Finished post open deploy task for testSplitTableRegionDeletedRowsDaughter,,1683694444835.e0a753dfc7b1edf68920ae1543edc38c. 2023-05-10 04:54:05,213 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionDeletedRowsDaughter,,1683694444835.e0a753dfc7b1edf68920ae1543edc38c. 2023-05-10 04:54:05,213 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=70 updating hbase:meta row=e0a753dfc7b1edf68920ae1543edc38c, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:05,213 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionDeletedRowsDaughter,,1683694444835.e0a753dfc7b1edf68920ae1543edc38c.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1683694445213"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1683694445213"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1683694445213"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683694445213"}]},"ts":"1683694445213"} 2023-05-10 04:54:05,216 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=71, resume processing ppid=70 2023-05-10 04:54:05,216 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=71, ppid=70, state=SUCCESS; OpenRegionProcedure e0a753dfc7b1edf68920ae1543edc38c, server=jenkins-hbase4.apache.org,37995,1683694419499 in 178 msec 2023-05-10 04:54:05,218 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=70, resume processing ppid=69 2023-05-10 04:54:05,218 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=70, ppid=69, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=e0a753dfc7b1edf68920ae1543edc38c, ASSIGN in 335 msec 2023-05-10 04:54:05,219 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-05-10 04:54:05,219 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683694445219"}]},"ts":"1683694445219"} 2023-05-10 04:54:05,220 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=ENABLED in hbase:meta 2023-05-10 04:54:05,224 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_POST_OPERATION 2023-05-10 04:54:05,225 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter in 389 msec 2023-05-10 04:54:05,345 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-10 04:54:05,346 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,347 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-05-10 04:54:05,347 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:54:05,347 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-05-10 04:54:05,347 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-05-10 04:54:05,347 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,347 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,349 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:54:05,349 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:54:05,349 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:54:05,349 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:54:05,349 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:54:05,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:54:05,349 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:05,349 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:54:05,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:54:05,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:54:05,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,349 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:05,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,349 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-10 04:54:05,350 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,350 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,350 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:54:05,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:54:05,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,350 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:54:05,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,350 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,350 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,350 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,351 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-05-10 04:54:05,351 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:54:05,351 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,351 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,351 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-05-10 04:54:05,351 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-05-10 04:54:05,351 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-05-10 04:54:05,351 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:54:05,351 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:54:05,351 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-05-10 04:54:05,351 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-10 04:54:05,351 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-10 04:54:05,351 DEBUG [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1683694444835.e0a753dfc7b1edf68920ae1543edc38c. 2023-05-10 04:54:05,351 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-05-10 04:54:05,351 DEBUG [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1683694444835.e0a753dfc7b1edf68920ae1543edc38c. started... 2023-05-10 04:54:05,351 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-05-10 04:54:05,351 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-05-10 04:54:05,351 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36665,1683694419452' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-05-10 04:54:05,351 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37823,1683694419534' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-05-10 04:54:05,352 INFO [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing e0a753dfc7b1edf68920ae1543edc38c 2/2 column families, dataSize=120 B heapSize=928 B 2023-05-10 04:54:05,354 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,355 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,355 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:54:05,355 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,363 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,363 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:54:05,372 INFO [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/.tmp/cf1/9c5dc820063c4ac1b94e51a619cf0ca3 2023-05-10 04:54:05,389 INFO [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/.tmp/cf2/8dc8ff2e94224e0bb3ca246e89c65dad 2023-05-10 04:54:05,393 DEBUG [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/.tmp/cf1/9c5dc820063c4ac1b94e51a619cf0ca3 as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf1/9c5dc820063c4ac1b94e51a619cf0ca3 2023-05-10 04:54:05,398 INFO [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf1/9c5dc820063c4ac1b94e51a619cf0ca3, entries=2, sequenceid=6, filesize=4.8 K 2023-05-10 04:54:05,398 DEBUG [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/.tmp/cf2/8dc8ff2e94224e0bb3ca246e89c65dad as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf2/8dc8ff2e94224e0bb3ca246e89c65dad 2023-05-10 04:54:05,402 INFO [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf2/8dc8ff2e94224e0bb3ca246e89c65dad, entries=2, sequenceid=6, filesize=4.8 K 2023-05-10 04:54:05,403 INFO [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for e0a753dfc7b1edf68920ae1543edc38c in 51ms, sequenceid=6, compaction requested=false 2023-05-10 04:54:05,403 DEBUG [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegionDeletedRowsDaughter' 2023-05-10 04:54:05,404 DEBUG [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for e0a753dfc7b1edf68920ae1543edc38c: 2023-05-10 04:54:05,404 DEBUG [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1683694444835.e0a753dfc7b1edf68920ae1543edc38c. 2023-05-10 04:54:05,404 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-10 04:54:05,404 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-10 04:54:05,404 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:05,404 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-05-10 04:54:05,404 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37995,1683694419499' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-05-10 04:54:05,412 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,412 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:05,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:05,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:54:05,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:54:05,412 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,412 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:54:05,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:54:05,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:54:05,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:54:05,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:05,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:54:05,414 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:54:05,414 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37995,1683694419499' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-05-10 04:54:05,414 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1df100ab[Count = 0] remaining members to acquire global barrier 2023-05-10 04:54:05,414 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-05-10 04:54:05,414 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,416 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,416 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,416 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,416 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,416 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-05-10 04:54:05,416 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:05,416 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-10 04:54:05,416 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,416 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-05-10 04:54:05,416 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,416 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,37995,1683694419499' in zk 2023-05-10 04:54:05,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,416 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-05-10 04:54:05,416 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-05-10 04:54:05,416 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-05-10 04:54:05,416 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-05-10 04:54:05,416 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,37823,1683694419534' in zk 2023-05-10 04:54:05,416 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,36665,1683694419452' in zk 2023-05-10 04:54:05,417 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-05-10 04:54:05,417 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:54:05,418 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:05,418 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-05-10 04:54:05,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:05,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:54:05,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:54:05,418 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-05-10 04:54:05,418 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-05-10 04:54:05,418 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:54:05,418 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:54:05,418 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:54:05,418 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-05-10 04:54:05,418 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-05-10 04:54:05,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:54:05,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:54:05,419 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:05,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:54:05,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:54:05,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:05,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:54:05,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:54:05,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,37995,1683694419499': 2023-05-10 04:54:05,421 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37995,1683694419499' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-05-10 04:54:05,421 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-05-10 04:54:05,421 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-10 04:54:05,421 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-10 04:54:05,421 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,421 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-10 04:54:05,426 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,426 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,426 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,426 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,426 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:54:05,426 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,426 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,427 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:54:05,427 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:54:05,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:54:05,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,427 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:54:05,426 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:54:05,427 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:54:05,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:54:05,427 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:54:05,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:54:05,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,427 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:54:05,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:54:05,427 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:05,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:54:05,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,428 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:54:05,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:54:05,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:54:05,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:05,429 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:05,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:54:05,429 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:54:05,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:54:05,430 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:54:05,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:05,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:54:05,431 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:54:05,437 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:54:05,437 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:54:05,437 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:54:05,437 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:54:05,437 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:54:05,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:54:05,437 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,437 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:54:05,437 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:54:05,438 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:54:05,438 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:54:05,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:54:05,438 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:54:05,438 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-05-10 04:54:05,438 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-10 04:54:05,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:54:05,438 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:05,437 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:54:05,438 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:54:05,438 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:54:05,438 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:54:05,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:54:05,438 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-05-10 04:54:05,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:54:05,438 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,438 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-10 04:54:05,438 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:54:05,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:54:05,438 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:54:05,439 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,439 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:54:05,439 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:05,439 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,439 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,539 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-10 04:54:05,540 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-10 04:54:05,553 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-10 04:54:05,554 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-05-10 04:54:05,554 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,554 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-05-10 04:54:05,554 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:54:05,554 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-05-10 04:54:05,554 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-05-10 04:54:05,555 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,555 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,557 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:54:05,557 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:54:05,557 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:54:05,557 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:54:05,557 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:54:05,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:54:05,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:54:05,557 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:05,557 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:54:05,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:54:05,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,557 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:05,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,557 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-10 04:54:05,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,558 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,558 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,558 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:54:05,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:54:05,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,558 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,558 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:54:05,558 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-05-10 04:54:05,558 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-05-10 04:54:05,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,558 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,558 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,559 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-05-10 04:54:05,559 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-05-10 04:54:05,559 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,559 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:54:05,559 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:54:05,559 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-05-10 04:54:05,559 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-05-10 04:54:05,559 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-05-10 04:54:05,559 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,559 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-05-10 04:54:05,559 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37823,1683694419534' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-05-10 04:54:05,559 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-05-10 04:54:05,559 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:54:05,559 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-05-10 04:54:05,559 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-05-10 04:54:05,559 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36665,1683694419452' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-05-10 04:54:05,560 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,561 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,561 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:54:05,561 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,562 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,562 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:54:05,562 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-10 04:54:05,562 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-10 04:54:05,562 DEBUG [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1683694444835.e0a753dfc7b1edf68920ae1543edc38c. 2023-05-10 04:54:05,563 DEBUG [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1683694444835.e0a753dfc7b1edf68920ae1543edc38c. started... 2023-05-10 04:54:05,563 INFO [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing e0a753dfc7b1edf68920ae1543edc38c 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-10 04:54:05,577 INFO [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/.tmp/cf1/5c92186c363e4e2ea144b06cf2ef4148 2023-05-10 04:54:05,593 INFO [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/.tmp/cf2/bda778111963494983a9d36e39fdf23e 2023-05-10 04:54:05,597 DEBUG [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/.tmp/cf1/5c92186c363e4e2ea144b06cf2ef4148 as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf1/5c92186c363e4e2ea144b06cf2ef4148 2023-05-10 04:54:05,603 INFO [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf1/5c92186c363e4e2ea144b06cf2ef4148, entries=10, sequenceid=19, filesize=5.0 K 2023-05-10 04:54:05,604 DEBUG [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/.tmp/cf2/bda778111963494983a9d36e39fdf23e as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf2/bda778111963494983a9d36e39fdf23e 2023-05-10 04:54:05,608 INFO [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf2/bda778111963494983a9d36e39fdf23e, entries=10, sequenceid=19, filesize=5.0 K 2023-05-10 04:54:05,609 INFO [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for e0a753dfc7b1edf68920ae1543edc38c in 45ms, sequenceid=19, compaction requested=false 2023-05-10 04:54:05,609 DEBUG [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for e0a753dfc7b1edf68920ae1543edc38c: 2023-05-10 04:54:05,609 DEBUG [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1683694444835.e0a753dfc7b1edf68920ae1543edc38c. 2023-05-10 04:54:05,609 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-10 04:54:05,609 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-10 04:54:05,609 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:05,609 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-05-10 04:54:05,609 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37995,1683694419499' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-05-10 04:54:05,612 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:05,612 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:05,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:54:05,612 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:54:05,613 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,613 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:54:05,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:54:05,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:54:05,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:54:05,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:05,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:54:05,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:54:05,615 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37995,1683694419499' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-05-10 04:54:05,615 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@14e671cf[Count = 0] remaining members to acquire global barrier 2023-05-10 04:54:05,615 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-05-10 04:54:05,615 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,616 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,616 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,616 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,616 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,616 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,617 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-05-10 04:54:05,617 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:05,617 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-10 04:54:05,617 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,617 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-05-10 04:54:05,617 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-05-10 04:54:05,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,617 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-05-10 04:54:05,617 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-05-10 04:54:05,617 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-05-10 04:54:05,617 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,37995,1683694419499' in zk 2023-05-10 04:54:05,617 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,36665,1683694419452' in zk 2023-05-10 04:54:05,617 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,37823,1683694419534' in zk 2023-05-10 04:54:05,619 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-05-10 04:54:05,619 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:54:05,619 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-05-10 04:54:05,620 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:05,620 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-05-10 04:54:05,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:05,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:54:05,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:54:05,620 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:54:05,620 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-05-10 04:54:05,620 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-05-10 04:54:05,620 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:54:05,620 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-05-10 04:54:05,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:54:05,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:54:05,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:54:05,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:05,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:54:05,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:54:05,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:54:05,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:05,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:54:05,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,37995,1683694419499': 2023-05-10 04:54:05,623 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37995,1683694419499' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-05-10 04:54:05,623 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-05-10 04:54:05,623 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-10 04:54:05,623 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-10 04:54:05,623 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,623 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-10 04:54:05,626 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,626 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,626 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,626 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,626 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:54:05,626 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,626 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:54:05,626 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,627 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:54:05,627 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:54:05,626 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:54:05,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:54:05,627 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:54:05,626 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:54:05,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:54:05,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:54:05,627 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:54:05,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:54:05,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,627 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:05,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:54:05,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,628 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:54:05,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:54:05,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:54:05,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:05,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:54:05,629 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:54:05,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:54:05,630 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:05,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,630 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:54:05,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:54:05,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:05,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:54:05,633 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:54:05,633 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:54:05,633 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:54:05,633 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,633 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:54:05,633 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:05,633 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:54:05,633 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:54:05,633 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:54:05,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:54:05,633 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:54:05,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:54:05,634 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:54:05,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:54:05,633 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:54:05,633 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-05-10 04:54:05,634 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:54:05,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:54:05,633 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:54:05,634 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:54:05,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:54:05,634 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-10 04:54:05,634 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,634 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:54:05,634 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:54:05,634 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:54:05,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:54:05,634 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,634 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-05-10 04:54:05,635 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:05,635 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-10 04:54:05,635 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:54:05,635 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,635 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,735 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-10 04:54:05,736 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-10 04:54:05,749 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-10 04:54:05,751 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-05-10 04:54:05,751 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,751 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-05-10 04:54:05,751 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:54:05,751 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-05-10 04:54:05,751 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-05-10 04:54:05,752 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,752 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,753 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:05,753 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:54:05,753 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:54:05,753 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:54:05,753 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:54:05,753 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:54:05,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:54:05,753 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:05,753 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-10 04:54:05,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:54:05,753 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:54:05,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:54:05,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,754 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,754 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,754 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:54:05,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,755 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:54:05,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:54:05,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,755 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,755 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,755 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-05-10 04:54:05,755 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,755 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-05-10 04:54:05,755 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-05-10 04:54:05,755 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-05-10 04:54:05,755 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:54:05,755 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,755 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,756 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-05-10 04:54:05,756 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:54:05,756 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-05-10 04:54:05,756 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:54:05,755 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-05-10 04:54:05,756 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-05-10 04:54:05,756 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36665,1683694419452' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-05-10 04:54:05,756 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-05-10 04:54:05,756 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-05-10 04:54:05,756 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37823,1683694419534' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-05-10 04:54:05,756 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-05-10 04:54:05,756 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-10 04:54:05,756 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-10 04:54:05,756 DEBUG [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1683694444835.e0a753dfc7b1edf68920ae1543edc38c. 2023-05-10 04:54:05,756 DEBUG [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1683694444835.e0a753dfc7b1edf68920ae1543edc38c. started... 2023-05-10 04:54:05,756 INFO [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing e0a753dfc7b1edf68920ae1543edc38c 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-10 04:54:05,758 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,760 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,760 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:54:05,760 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,760 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,761 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:54:05,767 INFO [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/.tmp/cf1/4391ef9079694b3d9c4aa820517a211f 2023-05-10 04:54:05,782 INFO [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/.tmp/cf2/c0c30c5caa4343a39ca186c265342dc5 2023-05-10 04:54:05,787 DEBUG [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/.tmp/cf1/4391ef9079694b3d9c4aa820517a211f as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf1/4391ef9079694b3d9c4aa820517a211f 2023-05-10 04:54:05,791 INFO [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf1/4391ef9079694b3d9c4aa820517a211f, entries=10, sequenceid=32, filesize=5.0 K 2023-05-10 04:54:05,792 DEBUG [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/.tmp/cf2/c0c30c5caa4343a39ca186c265342dc5 as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf2/c0c30c5caa4343a39ca186c265342dc5 2023-05-10 04:54:05,797 INFO [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf2/c0c30c5caa4343a39ca186c265342dc5, entries=10, sequenceid=32, filesize=5.0 K 2023-05-10 04:54:05,798 INFO [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for e0a753dfc7b1edf68920ae1543edc38c in 42ms, sequenceid=32, compaction requested=true 2023-05-10 04:54:05,798 DEBUG [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for e0a753dfc7b1edf68920ae1543edc38c: 2023-05-10 04:54:05,798 DEBUG [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1683694444835.e0a753dfc7b1edf68920ae1543edc38c. 2023-05-10 04:54:05,798 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-10 04:54:05,798 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-10 04:54:05,798 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:05,798 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-05-10 04:54:05,798 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37995,1683694419499' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-05-10 04:54:05,803 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:05,803 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:05,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:54:05,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:54:05,804 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:54:05,804 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:54:05,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:54:05,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:54:05,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:05,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:54:05,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:54:05,806 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37995,1683694419499' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-05-10 04:54:05,806 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7605fad3[Count = 0] remaining members to acquire global barrier 2023-05-10 04:54:05,806 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-05-10 04:54:05,806 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,807 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,807 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,807 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,807 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,807 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,807 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,807 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-05-10 04:54:05,807 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-05-10 04:54:05,808 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-05-10 04:54:05,808 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:05,808 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-10 04:54:05,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,808 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,37823,1683694419534' in zk 2023-05-10 04:54:05,808 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-05-10 04:54:05,808 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-05-10 04:54:05,808 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-05-10 04:54:05,808 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,36665,1683694419452' in zk 2023-05-10 04:54:05,808 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,37995,1683694419499' in zk 2023-05-10 04:54:05,810 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-05-10 04:54:05,810 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:54:05,810 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-05-10 04:54:05,812 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-05-10 04:54:05,812 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:05,812 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:54:05,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:05,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:54:05,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:54:05,812 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-05-10 04:54:05,812 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-05-10 04:54:05,813 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:54:05,813 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-05-10 04:54:05,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:54:05,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:54:05,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:54:05,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:05,814 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:54:05,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:54:05,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:54:05,815 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:05,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:54:05,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,37995,1683694419499': 2023-05-10 04:54:05,816 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37995,1683694419499' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-05-10 04:54:05,816 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-05-10 04:54:05,816 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-10 04:54:05,816 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-10 04:54:05,816 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,816 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-10 04:54:05,818 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,818 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,818 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:54:05,818 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,819 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,818 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:54:05,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:54:05,818 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,818 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:54:05,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,819 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,819 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:54:05,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,819 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:54:05,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:54:05,819 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:54:05,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:54:05,819 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:54:05,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:54:05,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:54:05,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,820 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:54:05,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,820 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:05,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:54:05,820 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:54:05,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:54:05,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:05,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:54:05,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:54:05,822 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:54:05,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,822 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:05,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:54:05,822 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:54:05,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:05,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:54:05,825 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:54:05,825 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:54:05,825 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:54:05,825 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:54:05,825 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-05-10 04:54:05,825 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-10 04:54:05,825 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:54:05,825 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:54:05,825 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,825 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:54:05,825 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:54:05,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:54:05,825 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:54:05,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:54:05,825 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-05-10 04:54:05,825 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:05,825 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:54:05,825 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:54:05,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:54:05,825 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:54:05,825 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-10 04:54:05,826 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,826 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:54:05,826 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:54:05,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:54:05,826 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:54:05,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:54:05,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:54:05,826 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:54:05,826 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,826 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:05,826 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:54:05,826 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,826 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,926 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-10 04:54:05,927 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-10 04:54:05,940 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-10 04:54:05,942 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-05-10 04:54:05,942 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,942 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-05-10 04:54:05,942 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:54:05,942 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-05-10 04:54:05,942 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-05-10 04:54:05,942 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,942 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,944 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:54:05,944 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:05,944 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:54:05,944 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:54:05,944 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:54:05,944 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:54:05,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:54:05,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:54:05,944 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:54:05,944 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:05,944 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-10 04:54:05,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:54:05,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,945 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,945 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,946 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:54:05,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,946 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,946 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-05-10 04:54:05,946 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:54:05,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,946 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-05-10 04:54:05,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:54:05,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,946 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:54:05,946 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,946 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-05-10 04:54:05,946 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,946 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-05-10 04:54:05,947 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36665,1683694419452' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-05-10 04:54:05,949 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,950 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,950 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:54:05,950 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-05-10 04:54:05,950 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,951 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-05-10 04:54:05,951 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-05-10 04:54:05,951 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,951 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:54:05,951 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-05-10 04:54:05,951 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:54:05,951 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-05-10 04:54:05,951 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-05-10 04:54:05,951 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-05-10 04:54:05,951 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37823,1683694419534' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-05-10 04:54:05,951 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-10 04:54:05,951 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-10 04:54:05,951 DEBUG [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1683694444835.e0a753dfc7b1edf68920ae1543edc38c. 2023-05-10 04:54:05,951 DEBUG [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1683694444835.e0a753dfc7b1edf68920ae1543edc38c. started... 2023-05-10 04:54:05,951 INFO [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing e0a753dfc7b1edf68920ae1543edc38c 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-10 04:54:05,953 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,953 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,953 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:54:05,966 INFO [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/.tmp/cf1/bee3c0539163404a9c5862a802d06fc8 2023-05-10 04:54:05,980 INFO [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/.tmp/cf2/fcb908e83d9d411482f19b7258cd7021 2023-05-10 04:54:05,984 DEBUG [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/.tmp/cf1/bee3c0539163404a9c5862a802d06fc8 as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf1/bee3c0539163404a9c5862a802d06fc8 2023-05-10 04:54:05,988 INFO [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf1/bee3c0539163404a9c5862a802d06fc8, entries=10, sequenceid=45, filesize=5.0 K 2023-05-10 04:54:05,989 DEBUG [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/.tmp/cf2/fcb908e83d9d411482f19b7258cd7021 as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf2/fcb908e83d9d411482f19b7258cd7021 2023-05-10 04:54:05,993 INFO [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf2/fcb908e83d9d411482f19b7258cd7021, entries=10, sequenceid=45, filesize=5.0 K 2023-05-10 04:54:05,994 INFO [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for e0a753dfc7b1edf68920ae1543edc38c in 42ms, sequenceid=45, compaction requested=true 2023-05-10 04:54:05,994 DEBUG [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for e0a753dfc7b1edf68920ae1543edc38c: 2023-05-10 04:54:05,994 DEBUG [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1683694444835.e0a753dfc7b1edf68920ae1543edc38c. 2023-05-10 04:54:05,994 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-10 04:54:05,994 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-10 04:54:05,994 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:05,994 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-05-10 04:54:05,994 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37995,1683694419499' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-05-10 04:54:05,996 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,996 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:05,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:05,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:54:05,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:54:05,996 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,996 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:54:05,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:54:05,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:54:05,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:05,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:54:05,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:05,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:54:05,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:54:05,999 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37995,1683694419499' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-05-10 04:54:05,999 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@405fa614[Count = 0] remaining members to acquire global barrier 2023-05-10 04:54:05,999 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-05-10 04:54:05,999 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,000 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,000 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,000 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,000 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,000 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,000 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:06,000 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-10 04:54:06,000 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,000 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-05-10 04:54:06,001 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-05-10 04:54:06,001 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-05-10 04:54:06,001 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-05-10 04:54:06,001 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,37995,1683694419499' in zk 2023-05-10 04:54:06,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,001 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,36665,1683694419452' in zk 2023-05-10 04:54:06,001 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-05-10 04:54:06,001 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-05-10 04:54:06,001 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,37823,1683694419534' in zk 2023-05-10 04:54:06,007 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-05-10 04:54:06,007 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:06,007 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:54:06,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:06,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:54:06,007 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:54:06,007 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-05-10 04:54:06,008 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-05-10 04:54:06,008 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-05-10 04:54:06,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:54:06,008 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:54:06,008 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:54:06,008 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-05-10 04:54:06,008 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-05-10 04:54:06,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:54:06,008 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:54:06,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:06,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:54:06,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:54:06,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:06,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:54:06,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:54:06,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,37995,1683694419499': 2023-05-10 04:54:06,011 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37995,1683694419499' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-05-10 04:54:06,011 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-05-10 04:54:06,011 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-10 04:54:06,011 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-10 04:54:06,011 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,011 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-10 04:54:06,017 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,017 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,017 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:54:06,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:54:06,017 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,017 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:54:06,017 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,017 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:54:06,017 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,017 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:54:06,017 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,017 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:54:06,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:54:06,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,017 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,018 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:54:06,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:54:06,018 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:54:06,018 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:54:06,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:54:06,018 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:06,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:54:06,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,018 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:54:06,018 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:54:06,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:54:06,019 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:06,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:54:06,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:54:06,020 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:06,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,021 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:54:06,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:06,021 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:54:06,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:54:06,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:54:06,024 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:54:06,024 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:54:06,024 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:54:06,024 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:54:06,024 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:54:06,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:54:06,024 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:54:06,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:54:06,024 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:54:06,024 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:54:06,024 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,024 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:54:06,024 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:06,024 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-05-10 04:54:06,024 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:54:06,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:54:06,024 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:54:06,024 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:54:06,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:54:06,024 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-10 04:54:06,024 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:54:06,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:54:06,024 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:54:06,025 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,025 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:54:06,025 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:54:06,025 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,025 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:54:06,025 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:54:06,025 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:06,025 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-05-10 04:54:06,025 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,025 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-10 04:54:06,025 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,125 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-10 04:54:06,126 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-10 04:54:06,139 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-10 04:54:06,140 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-05-10 04:54:06,140 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,140 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-05-10 04:54:06,140 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:54:06,140 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-05-10 04:54:06,140 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-05-10 04:54:06,141 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,141 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,142 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:54:06,142 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:54:06,142 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:54:06,142 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:54:06,142 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:54:06,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:54:06,142 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:54:06,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:54:06,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:54:06,142 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:06,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,143 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:06,143 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-10 04:54:06,143 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,143 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,143 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:54:06,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:54:06,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:54:06,143 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,144 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,144 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,144 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-05-10 04:54:06,144 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,144 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-05-10 04:54:06,144 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-05-10 04:54:06,144 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:54:06,144 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,144 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-05-10 04:54:06,144 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-05-10 04:54:06,144 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:54:06,144 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-05-10 04:54:06,144 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-10 04:54:06,144 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-10 04:54:06,144 DEBUG [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1683694444835.e0a753dfc7b1edf68920ae1543edc38c. 2023-05-10 04:54:06,144 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-05-10 04:54:06,144 DEBUG [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1683694444835.e0a753dfc7b1edf68920ae1543edc38c. started... 2023-05-10 04:54:06,144 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-05-10 04:54:06,144 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,144 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36665,1683694419452' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-05-10 04:54:06,144 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-05-10 04:54:06,144 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:54:06,144 INFO [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing e0a753dfc7b1edf68920ae1543edc38c 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-10 04:54:06,144 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-05-10 04:54:06,145 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-05-10 04:54:06,145 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37823,1683694419534' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-05-10 04:54:06,147 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,148 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,148 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,148 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:54:06,148 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,148 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:54:06,157 INFO [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/.tmp/cf1/6abf15e207cc4ef7bb40e895775d046c 2023-05-10 04:54:06,168 INFO [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/.tmp/cf2/839ff9ffc87b4074b7e77f7b6e45b9ce 2023-05-10 04:54:06,172 DEBUG [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/.tmp/cf1/6abf15e207cc4ef7bb40e895775d046c as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf1/6abf15e207cc4ef7bb40e895775d046c 2023-05-10 04:54:06,176 INFO [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf1/6abf15e207cc4ef7bb40e895775d046c, entries=10, sequenceid=58, filesize=5.0 K 2023-05-10 04:54:06,176 DEBUG [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/.tmp/cf2/839ff9ffc87b4074b7e77f7b6e45b9ce as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf2/839ff9ffc87b4074b7e77f7b6e45b9ce 2023-05-10 04:54:06,181 INFO [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf2/839ff9ffc87b4074b7e77f7b6e45b9ce, entries=10, sequenceid=58, filesize=5.0 K 2023-05-10 04:54:06,182 INFO [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for e0a753dfc7b1edf68920ae1543edc38c in 38ms, sequenceid=58, compaction requested=true 2023-05-10 04:54:06,182 DEBUG [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for e0a753dfc7b1edf68920ae1543edc38c: 2023-05-10 04:54:06,182 DEBUG [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1683694444835.e0a753dfc7b1edf68920ae1543edc38c. 2023-05-10 04:54:06,182 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-10 04:54:06,182 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-10 04:54:06,182 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:06,182 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-05-10 04:54:06,182 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37995,1683694419499' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-05-10 04:54:06,184 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:06,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:06,184 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:54:06,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:54:06,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:54:06,184 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,184 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:54:06,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:54:06,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:54:06,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:06,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:54:06,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:54:06,186 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37995,1683694419499' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-05-10 04:54:06,186 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@55be12ec[Count = 0] remaining members to acquire global barrier 2023-05-10 04:54:06,186 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-05-10 04:54:06,186 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,188 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,188 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,188 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,188 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,188 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,188 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,188 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-05-10 04:54:06,188 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:06,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,188 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-10 04:54:06,188 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-05-10 04:54:06,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,188 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,36665,1683694419452' in zk 2023-05-10 04:54:06,188 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-05-10 04:54:06,188 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-05-10 04:54:06,188 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,37823,1683694419534' in zk 2023-05-10 04:54:06,188 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-05-10 04:54:06,188 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-05-10 04:54:06,188 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,37995,1683694419499' in zk 2023-05-10 04:54:06,191 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-05-10 04:54:06,191 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-05-10 04:54:06,191 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:54:06,191 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:54:06,191 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-05-10 04:54:06,191 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-05-10 04:54:06,191 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:06,191 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-05-10 04:54:06,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:06,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:54:06,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:54:06,191 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:54:06,192 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-05-10 04:54:06,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:54:06,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:54:06,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,192 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:54:06,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:06,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:54:06,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:54:06,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:54:06,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:06,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:54:06,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,37995,1683694419499': 2023-05-10 04:54:06,194 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37995,1683694419499' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-05-10 04:54:06,195 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-05-10 04:54:06,195 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-10 04:54:06,195 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-10 04:54:06,195 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,195 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-10 04:54:06,196 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,196 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,196 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,196 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,197 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:54:06,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:54:06,197 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,197 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,197 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:54:06,196 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,196 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:54:06,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:54:06,197 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:54:06,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:54:06,197 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:54:06,197 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:54:06,197 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:54:06,197 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:54:06,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:54:06,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:54:06,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,198 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:06,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,198 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,199 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:54:06,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:54:06,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,199 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:54:06,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:06,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:54:06,200 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:54:06,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:54:06,200 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:06,200 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,201 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:54:06,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:54:06,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:06,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:54:06,204 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:54:06,204 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:54:06,204 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:54:06,204 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:54:06,204 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:54:06,204 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,204 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:54:06,204 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:54:06,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:54:06,204 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:54:06,204 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:54:06,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:54:06,204 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:54:06,204 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:54:06,204 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:54:06,204 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-05-10 04:54:06,205 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-10 04:54:06,204 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:06,204 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:54:06,205 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:54:06,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:54:06,205 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,205 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:54:06,205 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:54:06,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:54:06,205 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,205 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-05-10 04:54:06,205 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:54:06,205 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:54:06,205 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:06,205 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-10 04:54:06,205 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:54:06,205 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,205 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,305 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-10 04:54:06,306 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-10 04:54:06,318 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-10 04:54:06,320 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-05-10 04:54:06,320 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,320 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-05-10 04:54:06,320 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:54:06,320 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-05-10 04:54:06,320 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-05-10 04:54:06,320 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,321 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,326 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:54:06,326 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:54:06,326 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:54:06,326 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:54:06,326 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:06,326 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:54:06,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:54:06,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:54:06,326 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:54:06,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:54:06,327 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:06,327 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-10 04:54:06,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,327 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,327 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,327 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:54:06,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:54:06,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:54:06,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,328 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,328 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,328 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,328 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-05-10 04:54:06,328 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,328 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-05-10 04:54:06,328 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-05-10 04:54:06,328 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:54:06,328 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-05-10 04:54:06,328 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-05-10 04:54:06,328 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,328 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,328 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-05-10 04:54:06,328 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-05-10 04:54:06,328 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-10 04:54:06,328 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-10 04:54:06,328 DEBUG [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1683694444835.e0a753dfc7b1edf68920ae1543edc38c. 2023-05-10 04:54:06,328 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:54:06,329 DEBUG [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1683694444835.e0a753dfc7b1edf68920ae1543edc38c. started... 2023-05-10 04:54:06,328 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:54:06,329 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-05-10 04:54:06,329 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-05-10 04:54:06,329 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-05-10 04:54:06,329 INFO [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing e0a753dfc7b1edf68920ae1543edc38c 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-10 04:54:06,329 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37823,1683694419534' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-05-10 04:54:06,329 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-05-10 04:54:06,329 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36665,1683694419452' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-05-10 04:54:06,331 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,331 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,331 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,331 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:54:06,332 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,332 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:54:06,338 INFO [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/.tmp/cf1/069e6767796b4374972ea4f62b4268aa 2023-05-10 04:54:06,350 INFO [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/.tmp/cf2/e8a131ba8e924d2ab9e7dd5b50323810 2023-05-10 04:54:06,355 DEBUG [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/.tmp/cf1/069e6767796b4374972ea4f62b4268aa as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf1/069e6767796b4374972ea4f62b4268aa 2023-05-10 04:54:06,358 INFO [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf1/069e6767796b4374972ea4f62b4268aa, entries=10, sequenceid=71, filesize=5.0 K 2023-05-10 04:54:06,359 DEBUG [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/.tmp/cf2/e8a131ba8e924d2ab9e7dd5b50323810 as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf2/e8a131ba8e924d2ab9e7dd5b50323810 2023-05-10 04:54:06,362 INFO [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf2/e8a131ba8e924d2ab9e7dd5b50323810, entries=10, sequenceid=71, filesize=5.0 K 2023-05-10 04:54:06,363 INFO [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for e0a753dfc7b1edf68920ae1543edc38c in 34ms, sequenceid=71, compaction requested=true 2023-05-10 04:54:06,363 DEBUG [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for e0a753dfc7b1edf68920ae1543edc38c: 2023-05-10 04:54:06,363 DEBUG [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1683694444835.e0a753dfc7b1edf68920ae1543edc38c. 2023-05-10 04:54:06,364 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-10 04:54:06,364 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-10 04:54:06,364 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:06,364 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-05-10 04:54:06,364 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37995,1683694419499' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-05-10 04:54:06,366 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:06,366 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:06,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:54:06,366 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:54:06,367 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:54:06,367 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:54:06,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:54:06,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:54:06,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:06,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:54:06,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:54:06,368 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37995,1683694419499' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-05-10 04:54:06,368 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@72f1699d[Count = 0] remaining members to acquire global barrier 2023-05-10 04:54:06,368 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-05-10 04:54:06,368 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,370 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,370 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,370 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,370 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,370 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-05-10 04:54:06,370 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:06,370 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-10 04:54:06,370 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,370 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-05-10 04:54:06,370 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,370 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,37823,1683694419534' in zk 2023-05-10 04:54:06,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,370 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-05-10 04:54:06,370 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-05-10 04:54:06,370 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-05-10 04:54:06,370 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-05-10 04:54:06,370 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,37995,1683694419499' in zk 2023-05-10 04:54:06,370 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,36665,1683694419452' in zk 2023-05-10 04:54:06,372 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-05-10 04:54:06,372 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:54:06,372 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-05-10 04:54:06,372 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-05-10 04:54:06,372 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:54:06,372 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:06,372 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-05-10 04:54:06,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:06,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:54:06,373 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:54:06,376 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-05-10 04:54:06,376 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:54:06,376 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:54:06,376 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-05-10 04:54:06,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:54:06,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:54:06,377 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:06,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:54:06,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:54:06,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,378 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:54:06,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:06,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:54:06,379 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,37995,1683694419499': 2023-05-10 04:54:06,379 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37995,1683694419499' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-05-10 04:54:06,379 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-05-10 04:54:06,380 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-10 04:54:06,380 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-10 04:54:06,380 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,380 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-10 04:54:06,381 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,381 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,381 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,381 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:54:06,381 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,381 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,381 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,381 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:54:06,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,381 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,381 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:54:06,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,381 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:54:06,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:54:06,382 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:54:06,382 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:54:06,382 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:54:06,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:54:06,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:54:06,382 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:54:06,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:54:06,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:54:06,382 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:06,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,382 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,383 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:54:06,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:54:06,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,383 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:54:06,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:06,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:54:06,384 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:54:06,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:54:06,384 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:06,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,384 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:54:06,384 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:54:06,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:06,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:54:06,387 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:54:06,387 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,387 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:54:06,387 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:54:06,387 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:54:06,387 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:54:06,387 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:54:06,387 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:54:06,387 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:54:06,387 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:06,387 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:54:06,387 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:54:06,387 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:54:06,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:54:06,387 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-05-10 04:54:06,388 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-10 04:54:06,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:54:06,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:54:06,387 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,388 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:54:06,388 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,387 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:54:06,388 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:06,388 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-05-10 04:54:06,388 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:54:06,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:54:06,388 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:54:06,388 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:54:06,388 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:54:06,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:54:06,388 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-10 04:54:06,388 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:54:06,388 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,388 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,488 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-10 04:54:06,489 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-10 04:54:06,504 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-10 04:54:06,505 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-05-10 04:54:06,505 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,505 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-05-10 04:54:06,506 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:54:06,506 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-05-10 04:54:06,506 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-05-10 04:54:06,506 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,506 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,507 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:54:06,507 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:54:06,507 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:54:06,507 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:06,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:54:06,507 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:54:06,507 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:54:06,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:54:06,508 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:54:06,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:54:06,508 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:06,508 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-10 04:54:06,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,508 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,508 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,508 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,509 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:54:06,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:54:06,509 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,509 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:54:06,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,509 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-05-10 04:54:06,509 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,509 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-05-10 04:54:06,509 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,509 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,509 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-05-10 04:54:06,509 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-05-10 04:54:06,509 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-05-10 04:54:06,509 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:54:06,509 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:54:06,509 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,509 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-05-10 04:54:06,509 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-05-10 04:54:06,510 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-05-10 04:54:06,510 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37823,1683694419534' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-05-10 04:54:06,510 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-05-10 04:54:06,510 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:54:06,510 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-10 04:54:06,510 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-10 04:54:06,510 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-05-10 04:54:06,510 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-05-10 04:54:06,510 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36665,1683694419452' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-05-10 04:54:06,510 DEBUG [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1683694444835.e0a753dfc7b1edf68920ae1543edc38c. 2023-05-10 04:54:06,510 DEBUG [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1683694444835.e0a753dfc7b1edf68920ae1543edc38c. started... 2023-05-10 04:54:06,510 INFO [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing e0a753dfc7b1edf68920ae1543edc38c 2/2 column families, dataSize=530 B heapSize=2.33 KB 2023-05-10 04:54:06,512 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,513 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,513 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,513 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:54:06,514 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,514 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:54:06,520 INFO [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=265 B at sequenceid=83 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/.tmp/cf1/6fcee43348954f60b12fefd67f843fe5 2023-05-10 04:54:06,524 INFO [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6fcee43348954f60b12fefd67f843fe5 2023-05-10 04:54:06,532 INFO [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=265 B at sequenceid=83 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/.tmp/cf2/8335cbbc49cf41f1982b7afcfeae3b15 2023-05-10 04:54:06,536 INFO [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8335cbbc49cf41f1982b7afcfeae3b15 2023-05-10 04:54:06,536 DEBUG [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/.tmp/cf1/6fcee43348954f60b12fefd67f843fe5 as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf1/6fcee43348954f60b12fefd67f843fe5 2023-05-10 04:54:06,540 INFO [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6fcee43348954f60b12fefd67f843fe5 2023-05-10 04:54:06,540 INFO [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf1/6fcee43348954f60b12fefd67f843fe5, entries=9, sequenceid=83, filesize=5.1 K 2023-05-10 04:54:06,541 DEBUG [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/.tmp/cf2/8335cbbc49cf41f1982b7afcfeae3b15 as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf2/8335cbbc49cf41f1982b7afcfeae3b15 2023-05-10 04:54:06,545 INFO [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8335cbbc49cf41f1982b7afcfeae3b15 2023-05-10 04:54:06,545 INFO [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf2/8335cbbc49cf41f1982b7afcfeae3b15, entries=9, sequenceid=83, filesize=5.1 K 2023-05-10 04:54:06,545 INFO [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~530 B/530, heapSize ~2.30 KB/2352, currentSize=0 B/0 for e0a753dfc7b1edf68920ae1543edc38c in 35ms, sequenceid=83, compaction requested=true 2023-05-10 04:54:06,545 DEBUG [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for e0a753dfc7b1edf68920ae1543edc38c: 2023-05-10 04:54:06,546 DEBUG [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1683694444835.e0a753dfc7b1edf68920ae1543edc38c. 2023-05-10 04:54:06,546 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-10 04:54:06,546 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-10 04:54:06,546 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:06,546 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-05-10 04:54:06,546 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37995,1683694419499' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-05-10 04:54:06,547 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:06,547 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:06,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:54:06,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:54:06,548 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,548 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:54:06,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:54:06,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:54:06,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:54:06,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:06,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:54:06,549 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:54:06,549 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37995,1683694419499' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-05-10 04:54:06,549 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2b7bc672[Count = 0] remaining members to acquire global barrier 2023-05-10 04:54:06,549 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-05-10 04:54:06,550 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,551 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,551 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,551 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,551 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,551 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,551 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-05-10 04:54:06,551 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,551 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:06,551 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-10 04:54:06,551 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,551 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-05-10 04:54:06,551 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-05-10 04:54:06,551 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,36665,1683694419452' in zk 2023-05-10 04:54:06,551 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-05-10 04:54:06,551 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-05-10 04:54:06,551 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,37995,1683694419499' in zk 2023-05-10 04:54:06,551 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-05-10 04:54:06,551 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,37823,1683694419534' in zk 2023-05-10 04:54:06,554 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-05-10 04:54:06,554 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:54:06,554 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-05-10 04:54:06,557 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:06,557 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-05-10 04:54:06,557 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:54:06,557 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-05-10 04:54:06,557 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-05-10 04:54:06,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:06,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:54:06,557 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:54:06,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:54:06,557 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-05-10 04:54:06,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:54:06,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:54:06,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:54:06,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:06,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:54:06,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:54:06,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:06,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:54:06,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:54:06,560 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,37995,1683694419499': 2023-05-10 04:54:06,560 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37995,1683694419499' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-05-10 04:54:06,560 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-05-10 04:54:06,560 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-10 04:54:06,560 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-10 04:54:06,560 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,560 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-10 04:54:06,562 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,562 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,562 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,562 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,562 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,562 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:54:06,562 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,562 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:54:06,562 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,563 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:54:06,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:54:06,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:54:06,562 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,562 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:54:06,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,563 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:54:06,563 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:54:06,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:54:06,563 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:06,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:54:06,563 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:54:06,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:54:06,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:54:06,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,563 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:54:06,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:54:06,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:54:06,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:06,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:54:06,565 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:06,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:54:06,565 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:54:06,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,565 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:54:06,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:06,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:54:06,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:54:06,568 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:54:06,568 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,568 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:06,568 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:54:06,568 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:54:06,568 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:54:06,568 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:54:06,568 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:54:06,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:54:06,568 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,568 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-05-10 04:54:06,568 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-10 04:54:06,568 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:54:06,568 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:54:06,568 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:54:06,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:54:06,568 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:54:06,568 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:54:06,568 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,569 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-05-10 04:54:06,568 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:54:06,569 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:54:06,568 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:54:06,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:54:06,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:54:06,569 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-10 04:54:06,569 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:54:06,569 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:54:06,569 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:54:06,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:54:06,569 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:06,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:54:06,569 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,569 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,669 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-10 04:54:06,670 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-10 04:54:06,678 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-10 04:54:06,679 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-05-10 04:54:06,679 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,679 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-05-10 04:54:06,679 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:54:06,679 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-05-10 04:54:06,679 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-05-10 04:54:06,680 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,680 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,682 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:54:06,682 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:54:06,682 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:06,682 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:54:06,682 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:54:06,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:54:06,682 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:54:06,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:54:06,683 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:54:06,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:54:06,683 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:06,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,683 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-10 04:54:06,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,683 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,683 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:54:06,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,683 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:54:06,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,683 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,684 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:54:06,684 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-05-10 04:54:06,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,684 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-05-10 04:54:06,684 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,684 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,684 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-05-10 04:54:06,684 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:54:06,684 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,684 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-05-10 04:54:06,684 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-05-10 04:54:06,684 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:54:06,684 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-05-10 04:54:06,684 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36665,1683694419452' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-05-10 04:54:06,684 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-05-10 04:54:06,684 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-05-10 04:54:06,684 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,684 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-10 04:54:06,685 DEBUG [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1683694444835.e0a753dfc7b1edf68920ae1543edc38c. 2023-05-10 04:54:06,685 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-05-10 04:54:06,685 DEBUG [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1683694444835.e0a753dfc7b1edf68920ae1543edc38c. started... 2023-05-10 04:54:06,685 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-10 04:54:06,685 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:54:06,685 INFO [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing e0a753dfc7b1edf68920ae1543edc38c 2/2 column families, dataSize=250 B heapSize=1.52 KB 2023-05-10 04:54:06,685 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-05-10 04:54:06,685 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-05-10 04:54:06,685 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37823,1683694419534' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-05-10 04:54:06,687 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,687 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,687 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,687 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:54:06,687 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,687 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:54:06,698 INFO [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=91 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/.tmp/cf1/b9ae488cd59742d9bf602451afc29300 2023-05-10 04:54:06,705 INFO [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b9ae488cd59742d9bf602451afc29300 2023-05-10 04:54:06,713 INFO [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=91 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/.tmp/cf2/f70a46cf983c4f61bd80c8814f0701eb 2023-05-10 04:54:06,717 INFO [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f70a46cf983c4f61bd80c8814f0701eb 2023-05-10 04:54:06,718 DEBUG [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/.tmp/cf1/b9ae488cd59742d9bf602451afc29300 as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf1/b9ae488cd59742d9bf602451afc29300 2023-05-10 04:54:06,723 INFO [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b9ae488cd59742d9bf602451afc29300 2023-05-10 04:54:06,723 INFO [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf1/b9ae488cd59742d9bf602451afc29300, entries=5, sequenceid=91, filesize=5.0 K 2023-05-10 04:54:06,724 DEBUG [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/.tmp/cf2/f70a46cf983c4f61bd80c8814f0701eb as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf2/f70a46cf983c4f61bd80c8814f0701eb 2023-05-10 04:54:06,728 INFO [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f70a46cf983c4f61bd80c8814f0701eb 2023-05-10 04:54:06,728 INFO [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf2/f70a46cf983c4f61bd80c8814f0701eb, entries=5, sequenceid=91, filesize=5.0 K 2023-05-10 04:54:06,729 INFO [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~250 B/250, heapSize ~1.48 KB/1520, currentSize=0 B/0 for e0a753dfc7b1edf68920ae1543edc38c in 44ms, sequenceid=91, compaction requested=true 2023-05-10 04:54:06,729 DEBUG [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for e0a753dfc7b1edf68920ae1543edc38c: 2023-05-10 04:54:06,729 DEBUG [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1683694444835.e0a753dfc7b1edf68920ae1543edc38c. 2023-05-10 04:54:06,729 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-10 04:54:06,729 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-10 04:54:06,729 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:06,729 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-05-10 04:54:06,729 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37995,1683694419499' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-05-10 04:54:06,731 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,731 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:06,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:06,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:54:06,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:54:06,731 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,731 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:54:06,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:54:06,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:54:06,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:54:06,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:06,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:54:06,733 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:54:06,733 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37995,1683694419499' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-05-10 04:54:06,733 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1629ba0f[Count = 0] remaining members to acquire global barrier 2023-05-10 04:54:06,733 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-05-10 04:54:06,733 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,736 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,736 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,736 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,736 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,736 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,736 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:06,736 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-05-10 04:54:06,736 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-10 04:54:06,736 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,736 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-05-10 04:54:06,736 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,37823,1683694419534' in zk 2023-05-10 04:54:06,736 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-05-10 04:54:06,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,736 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-05-10 04:54:06,736 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-05-10 04:54:06,736 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-05-10 04:54:06,736 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,36665,1683694419452' in zk 2023-05-10 04:54:06,736 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,37995,1683694419499' in zk 2023-05-10 04:54:06,738 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-05-10 04:54:06,738 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:54:06,738 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-05-10 04:54:06,739 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-05-10 04:54:06,739 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-05-10 04:54:06,739 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:06,739 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:54:06,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:06,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:54:06,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:54:06,739 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:54:06,739 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-05-10 04:54:06,739 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-05-10 04:54:06,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:54:06,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:54:06,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:54:06,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:06,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:54:06,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:54:06,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:54:06,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:06,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:54:06,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,37995,1683694419499': 2023-05-10 04:54:06,742 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37995,1683694419499' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-05-10 04:54:06,742 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-05-10 04:54:06,742 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-10 04:54:06,742 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-10 04:54:06,742 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,742 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-10 04:54:06,744 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,744 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,744 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:54:06,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:54:06,744 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,744 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:54:06,744 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,744 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,744 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:54:06,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,744 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:54:06,744 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:54:06,744 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:54:06,744 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:54:06,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:54:06,744 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:54:06,745 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:54:06,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:54:06,745 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:06,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:54:06,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,745 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:54:06,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:54:06,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:54:06,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:06,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:54:06,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:54:06,747 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:54:06,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,747 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:06,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:54:06,747 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:54:06,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:06,748 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:54:06,757 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:54:06,757 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:54:06,757 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,757 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:54:06,757 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:06,757 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:54:06,757 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:54:06,757 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:54:06,757 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:54:06,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:54:06,757 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:54:06,757 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-05-10 04:54:06,757 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-10 04:54:06,757 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:54:06,757 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:54:06,758 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:54:06,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:54:06,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:54:06,758 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,758 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-05-10 04:54:06,758 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-10 04:54:06,757 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:54:06,758 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:54:06,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:54:06,757 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:54:06,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:54:06,758 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:54:06,758 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,758 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:06,758 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:54:06,758 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,758 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,760 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:54:06,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:54:06,858 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-10 04:54:06,859 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-10 04:54:06,867 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-10 04:54:06,868 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-05-10 04:54:06,868 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,869 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-05-10 04:54:06,869 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:54:06,869 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-05-10 04:54:06,869 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-05-10 04:54:06,869 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,869 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,870 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:54:06,871 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:06,871 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:54:06,871 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:54:06,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:54:06,871 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:54:06,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:54:06,871 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:54:06,871 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:06,871 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:54:06,871 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-10 04:54:06,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:54:06,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,872 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,872 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,872 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:54:06,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:54:06,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:54:06,872 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,872 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,872 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,872 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-05-10 04:54:06,872 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,872 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-05-10 04:54:06,872 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-05-10 04:54:06,872 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:54:06,872 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,872 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-05-10 04:54:06,872 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-05-10 04:54:06,872 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-05-10 04:54:06,873 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-05-10 04:54:06,873 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:54:06,872 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37823,1683694419534' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-05-10 04:54:06,873 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-05-10 04:54:06,873 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-05-10 04:54:06,873 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36665,1683694419452' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-05-10 04:54:06,873 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,873 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-05-10 04:54:06,873 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:54:06,873 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-05-10 04:54:06,873 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-10 04:54:06,873 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-10 04:54:06,873 DEBUG [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1683694444835.e0a753dfc7b1edf68920ae1543edc38c. 2023-05-10 04:54:06,873 DEBUG [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1683694444835.e0a753dfc7b1edf68920ae1543edc38c. started... 2023-05-10 04:54:06,873 INFO [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing e0a753dfc7b1edf68920ae1543edc38c 2/2 column families, dataSize=250 B heapSize=1.52 KB 2023-05-10 04:54:06,874 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,875 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,875 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:54:06,875 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,875 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,875 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:54:06,882 INFO [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=99 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/.tmp/cf1/4916dab2636c42c190c9d9d879485e00 2023-05-10 04:54:06,886 INFO [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4916dab2636c42c190c9d9d879485e00 2023-05-10 04:54:06,895 INFO [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=99 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/.tmp/cf2/8c9cb9ffa4ef41de964cc06282779ac0 2023-05-10 04:54:06,900 INFO [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8c9cb9ffa4ef41de964cc06282779ac0 2023-05-10 04:54:06,901 DEBUG [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/.tmp/cf1/4916dab2636c42c190c9d9d879485e00 as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf1/4916dab2636c42c190c9d9d879485e00 2023-05-10 04:54:06,905 INFO [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4916dab2636c42c190c9d9d879485e00 2023-05-10 04:54:06,905 INFO [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf1/4916dab2636c42c190c9d9d879485e00, entries=5, sequenceid=99, filesize=5.0 K 2023-05-10 04:54:06,906 DEBUG [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/.tmp/cf2/8c9cb9ffa4ef41de964cc06282779ac0 as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf2/8c9cb9ffa4ef41de964cc06282779ac0 2023-05-10 04:54:06,909 INFO [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8c9cb9ffa4ef41de964cc06282779ac0 2023-05-10 04:54:06,909 INFO [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf2/8c9cb9ffa4ef41de964cc06282779ac0, entries=5, sequenceid=99, filesize=5.0 K 2023-05-10 04:54:06,910 INFO [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~250 B/250, heapSize ~1.48 KB/1520, currentSize=0 B/0 for e0a753dfc7b1edf68920ae1543edc38c in 37ms, sequenceid=99, compaction requested=true 2023-05-10 04:54:06,910 DEBUG [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for e0a753dfc7b1edf68920ae1543edc38c: 2023-05-10 04:54:06,910 DEBUG [rs(jenkins-hbase4.apache.org,37995,1683694419499)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1683694444835.e0a753dfc7b1edf68920ae1543edc38c. 2023-05-10 04:54:06,910 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-10 04:54:06,910 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-10 04:54:06,910 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:06,910 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-05-10 04:54:06,910 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37995,1683694419499' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-05-10 04:54:06,913 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,913 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:06,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:06,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:54:06,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:54:06,913 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,913 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:54:06,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:54:06,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:54:06,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:54:06,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:06,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:54:06,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:54:06,915 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37995,1683694419499' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-05-10 04:54:06,915 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3325f354[Count = 0] remaining members to acquire global barrier 2023-05-10 04:54:06,915 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-05-10 04:54:06,915 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,916 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,916 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,916 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,916 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,916 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,916 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,916 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-05-10 04:54:06,916 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:06,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,916 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-10 04:54:06,916 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-05-10 04:54:06,916 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,916 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,37823,1683694419534' in zk 2023-05-10 04:54:06,916 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-05-10 04:54:06,917 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-05-10 04:54:06,917 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,36665,1683694419452' in zk 2023-05-10 04:54:06,916 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-05-10 04:54:06,917 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-05-10 04:54:06,917 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase4.apache.org,37995,1683694419499' in zk 2023-05-10 04:54:06,918 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-05-10 04:54:06,918 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-05-10 04:54:06,918 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:54:06,918 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:54:06,918 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-05-10 04:54:06,918 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-05-10 04:54:06,919 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-05-10 04:54:06,919 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:54:06,919 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:06,919 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-05-10 04:54:06,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:06,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:54:06,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:54:06,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:54:06,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:54:06,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:54:06,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:06,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:54:06,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:54:06,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:54:06,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:06,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:54:06,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase4.apache.org,37995,1683694419499': 2023-05-10 04:54:06,922 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37995,1683694419499' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-05-10 04:54:06,922 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-05-10 04:54:06,922 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-10 04:54:06,922 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-10 04:54:06,922 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,922 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-10 04:54:06,924 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,924 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,925 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,925 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:54:06,925 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:54:06,925 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,925 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:54:06,924 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:54:06,925 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,925 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:54:06,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:54:06,925 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:54:06,925 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:54:06,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:54:06,925 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:54:06,925 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:54:06,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:54:06,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:54:06,925 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:06,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,926 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:54:06,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:54:06,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:54:06,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:06,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:54:06,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:54:06,927 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:54:06,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,928 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:06,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:54:06,928 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:54:06,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:06,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:54:06,930 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:54:06,930 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:54:06,930 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:54:06,930 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:54:06,930 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:54:06,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:54:06,930 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,930 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:54:06,931 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-05-10 04:54:06,931 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-10 04:54:06,931 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:54:06,931 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:54:06,931 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:06,930 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:54:06,931 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:54:06,930 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:54:06,931 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:54:06,931 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:54:06,931 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-05-10 04:54:06,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:54:06,931 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:54:06,931 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:54:06,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:54:06,931 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-10 04:54:06,931 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,931 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:54:06,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:54:06,931 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:06,931 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:54:06,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:54:06,932 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:54:06,932 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:06,932 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:07,031 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-10 04:54:07,032 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-10 04:54:07,035 DEBUG [Listener at localhost/45007] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:07,036 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(1029): Stored pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=e0a753dfc7b1edf68920ae1543edc38c, daughterA=08557250442eebc7c86235cdf682b1a9, daughterB=e1dcbf091d2db6dbef8d701eb3d89914 2023-05-10 04:54:07,037 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=e0a753dfc7b1edf68920ae1543edc38c, daughterA=08557250442eebc7c86235cdf682b1a9, daughterB=e1dcbf091d2db6dbef8d701eb3d89914 2023-05-10 04:54:07,037 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=e0a753dfc7b1edf68920ae1543edc38c, daughterA=08557250442eebc7c86235cdf682b1a9, daughterB=e1dcbf091d2db6dbef8d701eb3d89914 2023-05-10 04:54:07,037 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=e0a753dfc7b1edf68920ae1543edc38c, daughterA=08557250442eebc7c86235cdf682b1a9, daughterB=e1dcbf091d2db6dbef8d701eb3d89914 2023-05-10 04:54:07,038 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=73, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=e0a753dfc7b1edf68920ae1543edc38c, UNASSIGN}] 2023-05-10 04:54:07,039 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=73, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=e0a753dfc7b1edf68920ae1543edc38c, UNASSIGN 2023-05-10 04:54:07,039 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=73 updating hbase:meta row=e0a753dfc7b1edf68920ae1543edc38c, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:07,040 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1683694444835.e0a753dfc7b1edf68920ae1543edc38c.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1683694447039"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683694447039"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683694447039"}]},"ts":"1683694447039"} 2023-05-10 04:54:07,041 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=74, ppid=73, state=RUNNABLE; CloseRegionProcedure e0a753dfc7b1edf68920ae1543edc38c, server=jenkins-hbase4.apache.org,37995,1683694419499}] 2023-05-10 04:54:07,192 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close e0a753dfc7b1edf68920ae1543edc38c 2023-05-10 04:54:07,193 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing e0a753dfc7b1edf68920ae1543edc38c, disabling compactions & flushes 2023-05-10 04:54:07,193 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionDeletedRowsDaughter,,1683694444835.e0a753dfc7b1edf68920ae1543edc38c. 2023-05-10 04:54:07,193 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,,1683694444835.e0a753dfc7b1edf68920ae1543edc38c. 2023-05-10 04:54:07,193 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,,1683694444835.e0a753dfc7b1edf68920ae1543edc38c. after waiting 0 ms 2023-05-10 04:54:07,193 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,,1683694444835.e0a753dfc7b1edf68920ae1543edc38c. 2023-05-10 04:54:07,193 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing e0a753dfc7b1edf68920ae1543edc38c 2/2 column families, dataSize=50 B heapSize=720 B 2023-05-10 04:54:07,202 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=25 B at sequenceid=103 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/.tmp/cf1/aab44d0818ef4a9791980635a3fffd5d 2023-05-10 04:54:07,208 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for aab44d0818ef4a9791980635a3fffd5d 2023-05-10 04:54:07,216 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=25 B at sequenceid=103 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/.tmp/cf2/57715c44583d443d9ee24a31f2e0c753 2023-05-10 04:54:07,220 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 57715c44583d443d9ee24a31f2e0c753 2023-05-10 04:54:07,220 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/.tmp/cf1/aab44d0818ef4a9791980635a3fffd5d as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf1/aab44d0818ef4a9791980635a3fffd5d 2023-05-10 04:54:07,224 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for aab44d0818ef4a9791980635a3fffd5d 2023-05-10 04:54:07,224 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf1/aab44d0818ef4a9791980635a3fffd5d, entries=1, sequenceid=103, filesize=4.9 K 2023-05-10 04:54:07,224 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/.tmp/cf2/57715c44583d443d9ee24a31f2e0c753 as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf2/57715c44583d443d9ee24a31f2e0c753 2023-05-10 04:54:07,228 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 57715c44583d443d9ee24a31f2e0c753 2023-05-10 04:54:07,228 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf2/57715c44583d443d9ee24a31f2e0c753, entries=1, sequenceid=103, filesize=4.9 K 2023-05-10 04:54:07,228 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~50 B/50, heapSize ~688 B/688, currentSize=0 B/0 for e0a753dfc7b1edf68920ae1543edc38c in 35ms, sequenceid=103, compaction requested=true 2023-05-10 04:54:07,233 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/recovered.edits/106.seqid, newMaxSeqId=106, maxSeqId=1 2023-05-10 04:54:07,234 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-10 04:54:07,234 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionDeletedRowsDaughter,,1683694444835.e0a753dfc7b1edf68920ae1543edc38c. 2023-05-10 04:54:07,234 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for e0a753dfc7b1edf68920ae1543edc38c: 2023-05-10 04:54:07,235 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed e0a753dfc7b1edf68920ae1543edc38c 2023-05-10 04:54:07,236 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=73 updating hbase:meta row=e0a753dfc7b1edf68920ae1543edc38c, regionState=CLOSED 2023-05-10 04:54:07,236 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,,1683694444835.e0a753dfc7b1edf68920ae1543edc38c.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1683694447236"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683694447236"}]},"ts":"1683694447236"} 2023-05-10 04:54:07,238 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=74, resume processing ppid=73 2023-05-10 04:54:07,238 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=74, ppid=73, state=SUCCESS; CloseRegionProcedure e0a753dfc7b1edf68920ae1543edc38c, server=jenkins-hbase4.apache.org,37995,1683694419499 in 196 msec 2023-05-10 04:54:07,240 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=73, resume processing ppid=72 2023-05-10 04:54:07,240 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=73, ppid=72, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=e0a753dfc7b1edf68920ae1543edc38c, UNASSIGN in 200 msec 2023-05-10 04:54:07,255 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=72 splitting 20 storefiles, region=e0a753dfc7b1edf68920ae1543edc38c, threads=16 2023-05-10 04:54:07,255 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf2/57715c44583d443d9ee24a31f2e0c753 for region: e0a753dfc7b1edf68920ae1543edc38c 2023-05-10 04:54:07,255 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf2/8335cbbc49cf41f1982b7afcfeae3b15 for region: e0a753dfc7b1edf68920ae1543edc38c 2023-05-10 04:54:07,255 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf2/839ff9ffc87b4074b7e77f7b6e45b9ce for region: e0a753dfc7b1edf68920ae1543edc38c 2023-05-10 04:54:07,258 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf2/8c9cb9ffa4ef41de964cc06282779ac0 for region: e0a753dfc7b1edf68920ae1543edc38c 2023-05-10 04:54:07,259 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf2/8dc8ff2e94224e0bb3ca246e89c65dad for region: e0a753dfc7b1edf68920ae1543edc38c 2023-05-10 04:54:07,259 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf2/bda778111963494983a9d36e39fdf23e for region: e0a753dfc7b1edf68920ae1543edc38c 2023-05-10 04:54:07,262 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf2/c0c30c5caa4343a39ca186c265342dc5 for region: e0a753dfc7b1edf68920ae1543edc38c 2023-05-10 04:54:07,263 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf2/e8a131ba8e924d2ab9e7dd5b50323810 for region: e0a753dfc7b1edf68920ae1543edc38c 2023-05-10 04:54:07,263 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf2/f70a46cf983c4f61bd80c8814f0701eb for region: e0a753dfc7b1edf68920ae1543edc38c 2023-05-10 04:54:07,263 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf2/fcb908e83d9d411482f19b7258cd7021 for region: e0a753dfc7b1edf68920ae1543edc38c 2023-05-10 04:54:07,263 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf1/069e6767796b4374972ea4f62b4268aa for region: e0a753dfc7b1edf68920ae1543edc38c 2023-05-10 04:54:07,267 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf1/4391ef9079694b3d9c4aa820517a211f for region: e0a753dfc7b1edf68920ae1543edc38c 2023-05-10 04:54:07,267 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf1/4916dab2636c42c190c9d9d879485e00 for region: e0a753dfc7b1edf68920ae1543edc38c 2023-05-10 04:54:07,267 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf1/5c92186c363e4e2ea144b06cf2ef4148 for region: e0a753dfc7b1edf68920ae1543edc38c 2023-05-10 04:54:07,267 DEBUG [StoreFileSplitter-pool-15] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf1/6fcee43348954f60b12fefd67f843fe5 for region: e0a753dfc7b1edf68920ae1543edc38c 2023-05-10 04:54:07,267 DEBUG [StoreFileSplitter-pool-14] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf1/6abf15e207cc4ef7bb40e895775d046c for region: e0a753dfc7b1edf68920ae1543edc38c 2023-05-10 04:54:07,273 INFO [StoreFileSplitter-pool-3] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8c9cb9ffa4ef41de964cc06282779ac0 2023-05-10 04:54:07,273 DEBUG [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf2/839ff9ffc87b4074b7e77f7b6e45b9ce, top=false 2023-05-10 04:54:07,273 INFO [StoreFileSplitter-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 57715c44583d443d9ee24a31f2e0c753 2023-05-10 04:54:07,275 DEBUG [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf2/fcb908e83d9d411482f19b7258cd7021, top=false 2023-05-10 04:54:07,276 INFO [StoreFileSplitter-pool-3] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8c9cb9ffa4ef41de964cc06282779ac0 2023-05-10 04:54:07,276 DEBUG [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf2/8c9cb9ffa4ef41de964cc06282779ac0, top=true 2023-05-10 04:54:07,286 DEBUG [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf2/e8a131ba8e924d2ab9e7dd5b50323810, top=false 2023-05-10 04:54:07,287 INFO [StoreFileSplitter-pool-8] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f70a46cf983c4f61bd80c8814f0701eb 2023-05-10 04:54:07,290 INFO [StoreFileSplitter-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8335cbbc49cf41f1982b7afcfeae3b15 2023-05-10 04:54:07,296 DEBUG [StoreFileSplitter-pool-14] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf1/6abf15e207cc4ef7bb40e895775d046c, top=false 2023-05-10 04:54:07,301 INFO [StoreFileSplitter-pool-12] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4916dab2636c42c190c9d9d879485e00 2023-05-10 04:54:07,313 INFO [StoreFileSplitter-pool-12] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4916dab2636c42c190c9d9d879485e00 2023-05-10 04:54:07,313 DEBUG [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf1/4916dab2636c42c190c9d9d879485e00, top=true 2023-05-10 04:54:07,325 INFO [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/.splits/e1dcbf091d2db6dbef8d701eb3d89914/cf2/testSplitTableRegionDeletedRowsDaughter=e0a753dfc7b1edf68920ae1543edc38c-8c9cb9ffa4ef41de964cc06282779ac0 for child: e1dcbf091d2db6dbef8d701eb3d89914, parent: e0a753dfc7b1edf68920ae1543edc38c 2023-05-10 04:54:07,325 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf2/8c9cb9ffa4ef41de964cc06282779ac0 for region: e0a753dfc7b1edf68920ae1543edc38c 2023-05-10 04:54:07,325 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf1/9c5dc820063c4ac1b94e51a619cf0ca3 for region: e0a753dfc7b1edf68920ae1543edc38c 2023-05-10 04:54:07,327 INFO [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/.splits/08557250442eebc7c86235cdf682b1a9/cf2/testSplitTableRegionDeletedRowsDaughter=e0a753dfc7b1edf68920ae1543edc38c-fcb908e83d9d411482f19b7258cd7021 for child: 08557250442eebc7c86235cdf682b1a9, parent: e0a753dfc7b1edf68920ae1543edc38c 2023-05-10 04:54:07,342 INFO [StoreFileSplitter-pool-14] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/.splits/08557250442eebc7c86235cdf682b1a9/cf1/testSplitTableRegionDeletedRowsDaughter=e0a753dfc7b1edf68920ae1543edc38c-6abf15e207cc4ef7bb40e895775d046c for child: 08557250442eebc7c86235cdf682b1a9, parent: e0a753dfc7b1edf68920ae1543edc38c 2023-05-10 04:54:07,343 INFO [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/.splits/e1dcbf091d2db6dbef8d701eb3d89914/cf1/testSplitTableRegionDeletedRowsDaughter=e0a753dfc7b1edf68920ae1543edc38c-4916dab2636c42c190c9d9d879485e00 for child: e1dcbf091d2db6dbef8d701eb3d89914, parent: e0a753dfc7b1edf68920ae1543edc38c 2023-05-10 04:54:07,343 INFO [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/.splits/08557250442eebc7c86235cdf682b1a9/cf2/testSplitTableRegionDeletedRowsDaughter=e0a753dfc7b1edf68920ae1543edc38c-839ff9ffc87b4074b7e77f7b6e45b9ce for child: 08557250442eebc7c86235cdf682b1a9, parent: e0a753dfc7b1edf68920ae1543edc38c 2023-05-10 04:54:07,343 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf1/4916dab2636c42c190c9d9d879485e00 for region: e0a753dfc7b1edf68920ae1543edc38c 2023-05-10 04:54:07,343 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf1/aab44d0818ef4a9791980635a3fffd5d for region: e0a753dfc7b1edf68920ae1543edc38c 2023-05-10 04:54:07,343 DEBUG [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf1/069e6767796b4374972ea4f62b4268aa, top=false 2023-05-10 04:54:07,344 INFO [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/.splits/08557250442eebc7c86235cdf682b1a9/cf2/testSplitTableRegionDeletedRowsDaughter=e0a753dfc7b1edf68920ae1543edc38c-e8a131ba8e924d2ab9e7dd5b50323810 for child: 08557250442eebc7c86235cdf682b1a9, parent: e0a753dfc7b1edf68920ae1543edc38c 2023-05-10 04:54:07,346 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf2/fcb908e83d9d411482f19b7258cd7021 for region: e0a753dfc7b1edf68920ae1543edc38c 2023-05-10 04:54:07,346 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf1/b9ae488cd59742d9bf602451afc29300 for region: e0a753dfc7b1edf68920ae1543edc38c 2023-05-10 04:54:07,348 DEBUG [StoreFileSplitter-pool-14] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf1/6abf15e207cc4ef7bb40e895775d046c for region: e0a753dfc7b1edf68920ae1543edc38c 2023-05-10 04:54:07,348 DEBUG [StoreFileSplitter-pool-14] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf1/bee3c0539163404a9c5862a802d06fc8 for region: e0a753dfc7b1edf68920ae1543edc38c 2023-05-10 04:54:07,350 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf2/e8a131ba8e924d2ab9e7dd5b50323810 for region: e0a753dfc7b1edf68920ae1543edc38c 2023-05-10 04:54:07,353 DEBUG [StoreFileSplitter-pool-14] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf1/bee3c0539163404a9c5862a802d06fc8, top=false 2023-05-10 04:54:07,366 INFO [StoreFileSplitter-pool-15] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6fcee43348954f60b12fefd67f843fe5 2023-05-10 04:54:07,373 INFO [StoreFileSplitter-pool-8] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f70a46cf983c4f61bd80c8814f0701eb 2023-05-10 04:54:07,373 DEBUG [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf2/f70a46cf983c4f61bd80c8814f0701eb, top=true 2023-05-10 04:54:07,375 INFO [StoreFileSplitter-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 57715c44583d443d9ee24a31f2e0c753 2023-05-10 04:54:07,375 DEBUG [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf2/57715c44583d443d9ee24a31f2e0c753, top=true 2023-05-10 04:54:07,382 INFO [StoreFileSplitter-pool-12] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for aab44d0818ef4a9791980635a3fffd5d 2023-05-10 04:54:07,383 INFO [StoreFileSplitter-pool-9] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b9ae488cd59742d9bf602451afc29300 2023-05-10 04:54:07,383 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf2/839ff9ffc87b4074b7e77f7b6e45b9ce for region: e0a753dfc7b1edf68920ae1543edc38c 2023-05-10 04:54:07,387 INFO [StoreFileSplitter-pool-12] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for aab44d0818ef4a9791980635a3fffd5d 2023-05-10 04:54:07,387 DEBUG [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf1/aab44d0818ef4a9791980635a3fffd5d, top=true 2023-05-10 04:54:07,387 INFO [StoreFileSplitter-pool-9] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b9ae488cd59742d9bf602451afc29300 2023-05-10 04:54:07,387 DEBUG [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf1/b9ae488cd59742d9bf602451afc29300, top=true 2023-05-10 04:54:07,397 INFO [StoreFileSplitter-pool-14] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/.splits/08557250442eebc7c86235cdf682b1a9/cf1/testSplitTableRegionDeletedRowsDaughter=e0a753dfc7b1edf68920ae1543edc38c-bee3c0539163404a9c5862a802d06fc8 for child: 08557250442eebc7c86235cdf682b1a9, parent: e0a753dfc7b1edf68920ae1543edc38c 2023-05-10 04:54:07,397 INFO [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/.splits/08557250442eebc7c86235cdf682b1a9/cf1/testSplitTableRegionDeletedRowsDaughter=e0a753dfc7b1edf68920ae1543edc38c-069e6767796b4374972ea4f62b4268aa for child: 08557250442eebc7c86235cdf682b1a9, parent: e0a753dfc7b1edf68920ae1543edc38c 2023-05-10 04:54:07,404 INFO [StoreFileSplitter-pool-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8335cbbc49cf41f1982b7afcfeae3b15 2023-05-10 04:54:07,405 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf1/069e6767796b4374972ea4f62b4268aa for region: e0a753dfc7b1edf68920ae1543edc38c 2023-05-10 04:54:07,406 DEBUG [StoreFileSplitter-pool-14] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf1/bee3c0539163404a9c5862a802d06fc8 for region: e0a753dfc7b1edf68920ae1543edc38c 2023-05-10 04:54:07,415 INFO [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/.splits/e1dcbf091d2db6dbef8d701eb3d89914/cf1/testSplitTableRegionDeletedRowsDaughter=e0a753dfc7b1edf68920ae1543edc38c-b9ae488cd59742d9bf602451afc29300 for child: e1dcbf091d2db6dbef8d701eb3d89914, parent: e0a753dfc7b1edf68920ae1543edc38c 2023-05-10 04:54:07,415 INFO [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/.splits/e1dcbf091d2db6dbef8d701eb3d89914/cf1/testSplitTableRegionDeletedRowsDaughter=e0a753dfc7b1edf68920ae1543edc38c-aab44d0818ef4a9791980635a3fffd5d for child: e1dcbf091d2db6dbef8d701eb3d89914, parent: e0a753dfc7b1edf68920ae1543edc38c 2023-05-10 04:54:07,415 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf1/b9ae488cd59742d9bf602451afc29300 for region: e0a753dfc7b1edf68920ae1543edc38c 2023-05-10 04:54:07,415 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf1/aab44d0818ef4a9791980635a3fffd5d for region: e0a753dfc7b1edf68920ae1543edc38c 2023-05-10 04:54:07,417 INFO [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/.splits/e1dcbf091d2db6dbef8d701eb3d89914/cf2/testSplitTableRegionDeletedRowsDaughter=e0a753dfc7b1edf68920ae1543edc38c-57715c44583d443d9ee24a31f2e0c753 for child: e1dcbf091d2db6dbef8d701eb3d89914, parent: e0a753dfc7b1edf68920ae1543edc38c 2023-05-10 04:54:07,417 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf2/57715c44583d443d9ee24a31f2e0c753 for region: e0a753dfc7b1edf68920ae1543edc38c 2023-05-10 04:54:07,419 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf2/c0c30c5caa4343a39ca186c265342dc5 for region: e0a753dfc7b1edf68920ae1543edc38c 2023-05-10 04:54:07,419 INFO [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/.splits/e1dcbf091d2db6dbef8d701eb3d89914/cf2/testSplitTableRegionDeletedRowsDaughter=e0a753dfc7b1edf68920ae1543edc38c-f70a46cf983c4f61bd80c8814f0701eb for child: e1dcbf091d2db6dbef8d701eb3d89914, parent: e0a753dfc7b1edf68920ae1543edc38c 2023-05-10 04:54:07,419 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf2/f70a46cf983c4f61bd80c8814f0701eb for region: e0a753dfc7b1edf68920ae1543edc38c 2023-05-10 04:54:07,422 INFO [StoreFileSplitter-pool-15] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6fcee43348954f60b12fefd67f843fe5 2023-05-10 04:54:07,424 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf1/5c92186c363e4e2ea144b06cf2ef4148 for region: e0a753dfc7b1edf68920ae1543edc38c 2023-05-10 04:54:07,439 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf2/8dc8ff2e94224e0bb3ca246e89c65dad for region: e0a753dfc7b1edf68920ae1543edc38c 2023-05-10 04:54:07,451 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf1/4391ef9079694b3d9c4aa820517a211f for region: e0a753dfc7b1edf68920ae1543edc38c 2023-05-10 04:54:07,455 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf2/bda778111963494983a9d36e39fdf23e for region: e0a753dfc7b1edf68920ae1543edc38c 2023-05-10 04:54:07,463 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf2/8335cbbc49cf41f1982b7afcfeae3b15 for region: e0a753dfc7b1edf68920ae1543edc38c 2023-05-10 04:54:07,463 DEBUG [StoreFileSplitter-pool-15] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf1/6fcee43348954f60b12fefd67f843fe5 for region: e0a753dfc7b1edf68920ae1543edc38c 2023-05-10 04:54:07,469 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf1/9c5dc820063c4ac1b94e51a619cf0ca3 for region: e0a753dfc7b1edf68920ae1543edc38c 2023-05-10 04:54:07,469 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=72 split storefiles for region e0a753dfc7b1edf68920ae1543edc38c Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-05-10 04:54:07,519 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/08557250442eebc7c86235cdf682b1a9/recovered.edits/106.seqid, newMaxSeqId=106, maxSeqId=-1 2023-05-10 04:54:07,521 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e1dcbf091d2db6dbef8d701eb3d89914/recovered.edits/106.seqid, newMaxSeqId=106, maxSeqId=-1 2023-05-10 04:54:07,523 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1683694444835.e0a753dfc7b1edf68920ae1543edc38c.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1683694447522"},{"qualifier":"splitA","vlen":75,"tag":[],"timestamp":"1683694447522"},{"qualifier":"splitB","vlen":75,"tag":[],"timestamp":"1683694447522"}]},"ts":"1683694447522"} 2023-05-10 04:54:07,523 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1683694447035.08557250442eebc7c86235cdf682b1a9.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1683694447522"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683694447522"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683694447522"}]},"ts":"1683694447522"} 2023-05-10 04:54:07,523 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,60,1683694447035.e1dcbf091d2db6dbef8d701eb3d89914.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1683694447522"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683694447522"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683694447522"}]},"ts":"1683694447522"} 2023-05-10 04:54:07,526 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=75, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=08557250442eebc7c86235cdf682b1a9, ASSIGN}, {pid=76, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=e1dcbf091d2db6dbef8d701eb3d89914, ASSIGN}] 2023-05-10 04:54:07,527 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=76, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=e1dcbf091d2db6dbef8d701eb3d89914, ASSIGN 2023-05-10 04:54:07,528 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=76, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=e1dcbf091d2db6dbef8d701eb3d89914, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,37995,1683694419499; forceNewPlan=false, retain=false 2023-05-10 04:54:07,528 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=75, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=08557250442eebc7c86235cdf682b1a9, ASSIGN 2023-05-10 04:54:07,529 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=75, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=08557250442eebc7c86235cdf682b1a9, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,37995,1683694419499; forceNewPlan=false, retain=false 2023-05-10 04:54:07,678 INFO [jenkins-hbase4:37903] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-05-10 04:54:07,680 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=75 updating hbase:meta row=08557250442eebc7c86235cdf682b1a9, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:07,680 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1683694447035.08557250442eebc7c86235cdf682b1a9.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1683694447680"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683694447680"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683694447680"}]},"ts":"1683694447680"} 2023-05-10 04:54:07,682 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=77, ppid=75, state=RUNNABLE; OpenRegionProcedure 08557250442eebc7c86235cdf682b1a9, server=jenkins-hbase4.apache.org,37995,1683694419499}] 2023-05-10 04:54:07,684 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=e1dcbf091d2db6dbef8d701eb3d89914, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:07,685 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,60,1683694447035.e1dcbf091d2db6dbef8d701eb3d89914.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1683694447684"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683694447684"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683694447684"}]},"ts":"1683694447684"} 2023-05-10 04:54:07,686 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=78, ppid=76, state=RUNNABLE; OpenRegionProcedure e1dcbf091d2db6dbef8d701eb3d89914, server=jenkins-hbase4.apache.org,37995,1683694419499}] 2023-05-10 04:54:07,746 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-05-10 04:54:07,836 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionDeletedRowsDaughter,,1683694447035.08557250442eebc7c86235cdf682b1a9. 2023-05-10 04:54:07,837 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 08557250442eebc7c86235cdf682b1a9, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1683694447035.08557250442eebc7c86235cdf682b1a9.', STARTKEY => '', ENDKEY => '60'} 2023-05-10 04:54:07,837 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-05-10 04:54:07,837 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionDeletedRowsDaughter 08557250442eebc7c86235cdf682b1a9 2023-05-10 04:54:07,837 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionDeletedRowsDaughter,,1683694447035.08557250442eebc7c86235cdf682b1a9.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-10 04:54:07,837 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 08557250442eebc7c86235cdf682b1a9 2023-05-10 04:54:07,837 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 08557250442eebc7c86235cdf682b1a9 2023-05-10 04:54:07,838 INFO [StoreOpener-08557250442eebc7c86235cdf682b1a9-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 08557250442eebc7c86235cdf682b1a9 2023-05-10 04:54:07,839 DEBUG [StoreOpener-08557250442eebc7c86235cdf682b1a9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/08557250442eebc7c86235cdf682b1a9/cf1 2023-05-10 04:54:07,839 DEBUG [StoreOpener-08557250442eebc7c86235cdf682b1a9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/08557250442eebc7c86235cdf682b1a9/cf1 2023-05-10 04:54:07,840 INFO [StoreOpener-08557250442eebc7c86235cdf682b1a9-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 08557250442eebc7c86235cdf682b1a9 columnFamilyName cf1 2023-05-10 04:54:07,850 DEBUG [StoreOpener-08557250442eebc7c86235cdf682b1a9-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/08557250442eebc7c86235cdf682b1a9/cf1/4391ef9079694b3d9c4aa820517a211f.e0a753dfc7b1edf68920ae1543edc38c->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf1/4391ef9079694b3d9c4aa820517a211f-bottom 2023-05-10 04:54:07,853 DEBUG [StoreOpener-08557250442eebc7c86235cdf682b1a9-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/08557250442eebc7c86235cdf682b1a9/cf1/5c92186c363e4e2ea144b06cf2ef4148.e0a753dfc7b1edf68920ae1543edc38c->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf1/5c92186c363e4e2ea144b06cf2ef4148-bottom 2023-05-10 04:54:07,856 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6fcee43348954f60b12fefd67f843fe5.e0a753dfc7b1edf68920ae1543edc38c 2023-05-10 04:54:07,858 DEBUG [StoreOpener-08557250442eebc7c86235cdf682b1a9-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/08557250442eebc7c86235cdf682b1a9/cf1/6fcee43348954f60b12fefd67f843fe5.e0a753dfc7b1edf68920ae1543edc38c->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf1/6fcee43348954f60b12fefd67f843fe5-bottom 2023-05-10 04:54:07,862 DEBUG [StoreOpener-08557250442eebc7c86235cdf682b1a9-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/08557250442eebc7c86235cdf682b1a9/cf1/9c5dc820063c4ac1b94e51a619cf0ca3.e0a753dfc7b1edf68920ae1543edc38c->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf1/9c5dc820063c4ac1b94e51a619cf0ca3-bottom 2023-05-10 04:54:07,865 DEBUG [StoreOpener-08557250442eebc7c86235cdf682b1a9-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/08557250442eebc7c86235cdf682b1a9/cf1/testSplitTableRegionDeletedRowsDaughter=e0a753dfc7b1edf68920ae1543edc38c-069e6767796b4374972ea4f62b4268aa 2023-05-10 04:54:07,868 DEBUG [StoreOpener-08557250442eebc7c86235cdf682b1a9-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/08557250442eebc7c86235cdf682b1a9/cf1/testSplitTableRegionDeletedRowsDaughter=e0a753dfc7b1edf68920ae1543edc38c-6abf15e207cc4ef7bb40e895775d046c 2023-05-10 04:54:07,871 DEBUG [StoreOpener-08557250442eebc7c86235cdf682b1a9-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/08557250442eebc7c86235cdf682b1a9/cf1/testSplitTableRegionDeletedRowsDaughter=e0a753dfc7b1edf68920ae1543edc38c-bee3c0539163404a9c5862a802d06fc8 2023-05-10 04:54:07,871 INFO [StoreOpener-08557250442eebc7c86235cdf682b1a9-1] regionserver.HStore(310): Store=08557250442eebc7c86235cdf682b1a9/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-10 04:54:07,871 INFO [StoreOpener-08557250442eebc7c86235cdf682b1a9-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 08557250442eebc7c86235cdf682b1a9 2023-05-10 04:54:07,872 DEBUG [StoreOpener-08557250442eebc7c86235cdf682b1a9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/08557250442eebc7c86235cdf682b1a9/cf2 2023-05-10 04:54:07,872 DEBUG [StoreOpener-08557250442eebc7c86235cdf682b1a9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/08557250442eebc7c86235cdf682b1a9/cf2 2023-05-10 04:54:07,873 INFO [StoreOpener-08557250442eebc7c86235cdf682b1a9-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 08557250442eebc7c86235cdf682b1a9 columnFamilyName cf2 2023-05-10 04:54:07,882 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8335cbbc49cf41f1982b7afcfeae3b15.e0a753dfc7b1edf68920ae1543edc38c 2023-05-10 04:54:07,883 DEBUG [StoreOpener-08557250442eebc7c86235cdf682b1a9-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/08557250442eebc7c86235cdf682b1a9/cf2/8335cbbc49cf41f1982b7afcfeae3b15.e0a753dfc7b1edf68920ae1543edc38c->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf2/8335cbbc49cf41f1982b7afcfeae3b15-bottom 2023-05-10 04:54:07,888 DEBUG [StoreOpener-08557250442eebc7c86235cdf682b1a9-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/08557250442eebc7c86235cdf682b1a9/cf2/8dc8ff2e94224e0bb3ca246e89c65dad.e0a753dfc7b1edf68920ae1543edc38c->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf2/8dc8ff2e94224e0bb3ca246e89c65dad-bottom 2023-05-10 04:54:07,892 DEBUG [StoreOpener-08557250442eebc7c86235cdf682b1a9-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/08557250442eebc7c86235cdf682b1a9/cf2/bda778111963494983a9d36e39fdf23e.e0a753dfc7b1edf68920ae1543edc38c->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf2/bda778111963494983a9d36e39fdf23e-bottom 2023-05-10 04:54:07,896 DEBUG [StoreOpener-08557250442eebc7c86235cdf682b1a9-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/08557250442eebc7c86235cdf682b1a9/cf2/c0c30c5caa4343a39ca186c265342dc5.e0a753dfc7b1edf68920ae1543edc38c->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf2/c0c30c5caa4343a39ca186c265342dc5-bottom 2023-05-10 04:54:07,899 DEBUG [StoreOpener-08557250442eebc7c86235cdf682b1a9-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/08557250442eebc7c86235cdf682b1a9/cf2/testSplitTableRegionDeletedRowsDaughter=e0a753dfc7b1edf68920ae1543edc38c-839ff9ffc87b4074b7e77f7b6e45b9ce 2023-05-10 04:54:07,901 DEBUG [StoreOpener-08557250442eebc7c86235cdf682b1a9-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/08557250442eebc7c86235cdf682b1a9/cf2/testSplitTableRegionDeletedRowsDaughter=e0a753dfc7b1edf68920ae1543edc38c-e8a131ba8e924d2ab9e7dd5b50323810 2023-05-10 04:54:07,904 DEBUG [StoreOpener-08557250442eebc7c86235cdf682b1a9-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/08557250442eebc7c86235cdf682b1a9/cf2/testSplitTableRegionDeletedRowsDaughter=e0a753dfc7b1edf68920ae1543edc38c-fcb908e83d9d411482f19b7258cd7021 2023-05-10 04:54:07,905 INFO [StoreOpener-08557250442eebc7c86235cdf682b1a9-1] regionserver.HStore(310): Store=08557250442eebc7c86235cdf682b1a9/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-10 04:54:07,905 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/08557250442eebc7c86235cdf682b1a9 2023-05-10 04:54:07,906 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/08557250442eebc7c86235cdf682b1a9 2023-05-10 04:54:07,907 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionDeletedRowsDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-05-10 04:54:07,909 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 08557250442eebc7c86235cdf682b1a9 2023-05-10 04:54:07,909 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 08557250442eebc7c86235cdf682b1a9; next sequenceid=107; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70851570, jitterRate=0.055770665407180786}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-05-10 04:54:07,909 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 08557250442eebc7c86235cdf682b1a9: 2023-05-10 04:54:07,910 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2334): Post open deploy tasks for testSplitTableRegionDeletedRowsDaughter,,1683694447035.08557250442eebc7c86235cdf682b1a9., pid=77, masterSystemTime=1683694447833 2023-05-10 04:54:07,910 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-10 04:54:07,910 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-05-10 04:54:07,911 DEBUG [RS:1;jenkins-hbase4:37995-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-05-10 04:54:07,911 DEBUG [RS:1;jenkins-hbase4:37995-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-05-10 04:54:07,912 INFO [RS:1;jenkins-hbase4:37995-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,,1683694447035.08557250442eebc7c86235cdf682b1a9. 2023-05-10 04:54:07,912 DEBUG [RS:1;jenkins-hbase4:37995-shortCompactions-0] regionserver.HStore(1912): 08557250442eebc7c86235cdf682b1a9/cf1 is initiating minor compaction (all files) 2023-05-10 04:54:07,912 INFO [RS:1;jenkins-hbase4:37995-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,,1683694447035.08557250442eebc7c86235cdf682b1a9. 2023-05-10 04:54:07,912 INFO [RS:1;jenkins-hbase4:37995-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 08557250442eebc7c86235cdf682b1a9/cf1 in testSplitTableRegionDeletedRowsDaughter,,1683694447035.08557250442eebc7c86235cdf682b1a9. 2023-05-10 04:54:07,912 DEBUG [RS:1;jenkins-hbase4:37995-longCompactions-0] regionserver.HStore(1912): 08557250442eebc7c86235cdf682b1a9/cf2 is initiating minor compaction (all files) 2023-05-10 04:54:07,912 INFO [RS:1;jenkins-hbase4:37995-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 08557250442eebc7c86235cdf682b1a9/cf2 in testSplitTableRegionDeletedRowsDaughter,,1683694447035.08557250442eebc7c86235cdf682b1a9. 2023-05-10 04:54:07,912 INFO [RS:1;jenkins-hbase4:37995-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/08557250442eebc7c86235cdf682b1a9/cf1/9c5dc820063c4ac1b94e51a619cf0ca3.e0a753dfc7b1edf68920ae1543edc38c->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf1/9c5dc820063c4ac1b94e51a619cf0ca3-bottom, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/08557250442eebc7c86235cdf682b1a9/cf1/5c92186c363e4e2ea144b06cf2ef4148.e0a753dfc7b1edf68920ae1543edc38c->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf1/5c92186c363e4e2ea144b06cf2ef4148-bottom, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/08557250442eebc7c86235cdf682b1a9/cf1/4391ef9079694b3d9c4aa820517a211f.e0a753dfc7b1edf68920ae1543edc38c->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf1/4391ef9079694b3d9c4aa820517a211f-bottom, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/08557250442eebc7c86235cdf682b1a9/cf1/testSplitTableRegionDeletedRowsDaughter=e0a753dfc7b1edf68920ae1543edc38c-bee3c0539163404a9c5862a802d06fc8, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/08557250442eebc7c86235cdf682b1a9/cf1/testSplitTableRegionDeletedRowsDaughter=e0a753dfc7b1edf68920ae1543edc38c-6abf15e207cc4ef7bb40e895775d046c, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/08557250442eebc7c86235cdf682b1a9/cf1/testSplitTableRegionDeletedRowsDaughter=e0a753dfc7b1edf68920ae1543edc38c-069e6767796b4374972ea4f62b4268aa, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/08557250442eebc7c86235cdf682b1a9/cf1/6fcee43348954f60b12fefd67f843fe5.e0a753dfc7b1edf68920ae1543edc38c->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf1/6fcee43348954f60b12fefd67f843fe5-bottom] into tmpdir=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/08557250442eebc7c86235cdf682b1a9/.tmp, totalSize=35.0 K 2023-05-10 04:54:07,912 INFO [RS:1;jenkins-hbase4:37995-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/08557250442eebc7c86235cdf682b1a9/cf2/8dc8ff2e94224e0bb3ca246e89c65dad.e0a753dfc7b1edf68920ae1543edc38c->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf2/8dc8ff2e94224e0bb3ca246e89c65dad-bottom, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/08557250442eebc7c86235cdf682b1a9/cf2/bda778111963494983a9d36e39fdf23e.e0a753dfc7b1edf68920ae1543edc38c->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf2/bda778111963494983a9d36e39fdf23e-bottom, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/08557250442eebc7c86235cdf682b1a9/cf2/c0c30c5caa4343a39ca186c265342dc5.e0a753dfc7b1edf68920ae1543edc38c->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf2/c0c30c5caa4343a39ca186c265342dc5-bottom, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/08557250442eebc7c86235cdf682b1a9/cf2/testSplitTableRegionDeletedRowsDaughter=e0a753dfc7b1edf68920ae1543edc38c-fcb908e83d9d411482f19b7258cd7021, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/08557250442eebc7c86235cdf682b1a9/cf2/testSplitTableRegionDeletedRowsDaughter=e0a753dfc7b1edf68920ae1543edc38c-839ff9ffc87b4074b7e77f7b6e45b9ce, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/08557250442eebc7c86235cdf682b1a9/cf2/testSplitTableRegionDeletedRowsDaughter=e0a753dfc7b1edf68920ae1543edc38c-e8a131ba8e924d2ab9e7dd5b50323810, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/08557250442eebc7c86235cdf682b1a9/cf2/8335cbbc49cf41f1982b7afcfeae3b15.e0a753dfc7b1edf68920ae1543edc38c->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf2/8335cbbc49cf41f1982b7afcfeae3b15-bottom] into tmpdir=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/08557250442eebc7c86235cdf682b1a9/.tmp, totalSize=35.0 K 2023-05-10 04:54:07,913 DEBUG [RS:1;jenkins-hbase4:37995-shortCompactions-0] compactions.Compactor(207): Compacting 9c5dc820063c4ac1b94e51a619cf0ca3.e0a753dfc7b1edf68920ae1543edc38c, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1683694445340 2023-05-10 04:54:07,913 DEBUG [RS:1;jenkins-hbase4:37995-longCompactions-0] compactions.Compactor(207): Compacting 8dc8ff2e94224e0bb3ca246e89c65dad.e0a753dfc7b1edf68920ae1543edc38c, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1683694445340 2023-05-10 04:54:07,913 DEBUG [RS:1;jenkins-hbase4:37995-shortCompactions-0] compactions.Compactor(207): Compacting 5c92186c363e4e2ea144b06cf2ef4148.e0a753dfc7b1edf68920ae1543edc38c, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1683694445540 2023-05-10 04:54:07,913 DEBUG [RS:1;jenkins-hbase4:37995-longCompactions-0] compactions.Compactor(207): Compacting bda778111963494983a9d36e39fdf23e.e0a753dfc7b1edf68920ae1543edc38c, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1683694445540 2023-05-10 04:54:07,914 DEBUG [RS:1;jenkins-hbase4:37995-shortCompactions-0] compactions.Compactor(207): Compacting 4391ef9079694b3d9c4aa820517a211f.e0a753dfc7b1edf68920ae1543edc38c, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1683694445737 2023-05-10 04:54:07,914 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2361): Finished post open deploy task for testSplitTableRegionDeletedRowsDaughter,,1683694447035.08557250442eebc7c86235cdf682b1a9. 2023-05-10 04:54:07,914 DEBUG [RS:1;jenkins-hbase4:37995-longCompactions-0] compactions.Compactor(207): Compacting c0c30c5caa4343a39ca186c265342dc5.e0a753dfc7b1edf68920ae1543edc38c, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1683694445737 2023-05-10 04:54:07,914 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionDeletedRowsDaughter,,1683694447035.08557250442eebc7c86235cdf682b1a9. 2023-05-10 04:54:07,914 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionDeletedRowsDaughter,60,1683694447035.e1dcbf091d2db6dbef8d701eb3d89914. 2023-05-10 04:54:07,914 DEBUG [RS:1;jenkins-hbase4:37995-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=e0a753dfc7b1edf68920ae1543edc38c-bee3c0539163404a9c5862a802d06fc8, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1683694445928 2023-05-10 04:54:07,914 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => e1dcbf091d2db6dbef8d701eb3d89914, NAME => 'testSplitTableRegionDeletedRowsDaughter,60,1683694447035.e1dcbf091d2db6dbef8d701eb3d89914.', STARTKEY => '60', ENDKEY => ''} 2023-05-10 04:54:07,914 DEBUG [RS:1;jenkins-hbase4:37995-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=e0a753dfc7b1edf68920ae1543edc38c-fcb908e83d9d411482f19b7258cd7021, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1683694445928 2023-05-10 04:54:07,915 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=75 updating hbase:meta row=08557250442eebc7c86235cdf682b1a9, regionState=OPEN, openSeqNum=107, regionLocation=jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:07,915 DEBUG [RS:1;jenkins-hbase4:37995-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=e0a753dfc7b1edf68920ae1543edc38c-6abf15e207cc4ef7bb40e895775d046c, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1683694446127 2023-05-10 04:54:07,915 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionDeletedRowsDaughter,,1683694447035.08557250442eebc7c86235cdf682b1a9.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1683694447914"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1683694447914"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1683694447914"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683694447914"}]},"ts":"1683694447914"} 2023-05-10 04:54:07,915 DEBUG [RS:1;jenkins-hbase4:37995-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=e0a753dfc7b1edf68920ae1543edc38c-839ff9ffc87b4074b7e77f7b6e45b9ce, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1683694446127 2023-05-10 04:54:07,915 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-05-10 04:54:07,915 DEBUG [RS:1;jenkins-hbase4:37995-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=e0a753dfc7b1edf68920ae1543edc38c-069e6767796b4374972ea4f62b4268aa, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1683694446307 2023-05-10 04:54:07,915 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionDeletedRowsDaughter e1dcbf091d2db6dbef8d701eb3d89914 2023-05-10 04:54:07,915 DEBUG [RS:1;jenkins-hbase4:37995-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=e0a753dfc7b1edf68920ae1543edc38c-e8a131ba8e924d2ab9e7dd5b50323810, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1683694446307 2023-05-10 04:54:07,915 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionDeletedRowsDaughter,60,1683694447035.e1dcbf091d2db6dbef8d701eb3d89914.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-10 04:54:07,916 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for e1dcbf091d2db6dbef8d701eb3d89914 2023-05-10 04:54:07,916 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for e1dcbf091d2db6dbef8d701eb3d89914 2023-05-10 04:54:07,916 DEBUG [RS:1;jenkins-hbase4:37995-shortCompactions-0] compactions.Compactor(207): Compacting 6fcee43348954f60b12fefd67f843fe5.e0a753dfc7b1edf68920ae1543edc38c, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1683694446490 2023-05-10 04:54:07,916 DEBUG [RS:1;jenkins-hbase4:37995-longCompactions-0] compactions.Compactor(207): Compacting 8335cbbc49cf41f1982b7afcfeae3b15.e0a753dfc7b1edf68920ae1543edc38c, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1683694446490 2023-05-10 04:54:07,918 INFO [StoreOpener-e1dcbf091d2db6dbef8d701eb3d89914-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region e1dcbf091d2db6dbef8d701eb3d89914 2023-05-10 04:54:07,918 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=77, resume processing ppid=75 2023-05-10 04:54:07,919 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=77, ppid=75, state=SUCCESS; OpenRegionProcedure 08557250442eebc7c86235cdf682b1a9, server=jenkins-hbase4.apache.org,37995,1683694419499 in 235 msec 2023-05-10 04:54:07,920 DEBUG [StoreOpener-e1dcbf091d2db6dbef8d701eb3d89914-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e1dcbf091d2db6dbef8d701eb3d89914/cf1 2023-05-10 04:54:07,920 DEBUG [StoreOpener-e1dcbf091d2db6dbef8d701eb3d89914-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e1dcbf091d2db6dbef8d701eb3d89914/cf1 2023-05-10 04:54:07,920 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=75, ppid=72, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=08557250442eebc7c86235cdf682b1a9, ASSIGN in 393 msec 2023-05-10 04:54:07,920 INFO [StoreOpener-e1dcbf091d2db6dbef8d701eb3d89914-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e1dcbf091d2db6dbef8d701eb3d89914 columnFamilyName cf1 2023-05-10 04:54:07,934 INFO [RS:1;jenkins-hbase4:37995-longCompactions-0] throttle.PressureAwareThroughputController(145): 08557250442eebc7c86235cdf682b1a9#cf2#compaction#72 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-05-10 04:54:07,935 INFO [RS:1;jenkins-hbase4:37995-shortCompactions-0] throttle.PressureAwareThroughputController(145): 08557250442eebc7c86235cdf682b1a9#cf1#compaction#73 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-05-10 04:54:07,938 DEBUG [StoreOpener-e1dcbf091d2db6dbef8d701eb3d89914-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e1dcbf091d2db6dbef8d701eb3d89914/cf1/4391ef9079694b3d9c4aa820517a211f.e0a753dfc7b1edf68920ae1543edc38c->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf1/4391ef9079694b3d9c4aa820517a211f-top 2023-05-10 04:54:07,940 DEBUG [StoreOpener-e1dcbf091d2db6dbef8d701eb3d89914-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e1dcbf091d2db6dbef8d701eb3d89914/cf1/5c92186c363e4e2ea144b06cf2ef4148.e0a753dfc7b1edf68920ae1543edc38c->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf1/5c92186c363e4e2ea144b06cf2ef4148-top 2023-05-10 04:54:07,944 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6fcee43348954f60b12fefd67f843fe5.e0a753dfc7b1edf68920ae1543edc38c 2023-05-10 04:54:07,944 DEBUG [StoreOpener-e1dcbf091d2db6dbef8d701eb3d89914-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e1dcbf091d2db6dbef8d701eb3d89914/cf1/6fcee43348954f60b12fefd67f843fe5.e0a753dfc7b1edf68920ae1543edc38c->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf1/6fcee43348954f60b12fefd67f843fe5-top 2023-05-10 04:54:07,947 DEBUG [StoreOpener-e1dcbf091d2db6dbef8d701eb3d89914-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e1dcbf091d2db6dbef8d701eb3d89914/cf1/9c5dc820063c4ac1b94e51a619cf0ca3.e0a753dfc7b1edf68920ae1543edc38c->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf1/9c5dc820063c4ac1b94e51a619cf0ca3-top 2023-05-10 04:54:07,959 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 4916dab2636c42c190c9d9d879485e00 2023-05-10 04:54:07,959 DEBUG [StoreOpener-e1dcbf091d2db6dbef8d701eb3d89914-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e1dcbf091d2db6dbef8d701eb3d89914/cf1/testSplitTableRegionDeletedRowsDaughter=e0a753dfc7b1edf68920ae1543edc38c-4916dab2636c42c190c9d9d879485e00 2023-05-10 04:54:07,963 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for aab44d0818ef4a9791980635a3fffd5d 2023-05-10 04:54:07,963 DEBUG [StoreOpener-e1dcbf091d2db6dbef8d701eb3d89914-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e1dcbf091d2db6dbef8d701eb3d89914/cf1/testSplitTableRegionDeletedRowsDaughter=e0a753dfc7b1edf68920ae1543edc38c-aab44d0818ef4a9791980635a3fffd5d 2023-05-10 04:54:07,964 DEBUG [RS:1;jenkins-hbase4:37995-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/08557250442eebc7c86235cdf682b1a9/.tmp/cf2/09853ed488394ed1906541ea36b30999 as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/08557250442eebc7c86235cdf682b1a9/cf2/09853ed488394ed1906541ea36b30999 2023-05-10 04:54:07,964 DEBUG [RS:1;jenkins-hbase4:37995-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/08557250442eebc7c86235cdf682b1a9/.tmp/cf1/bec5fc19800e451dbe3124fb24f21c54 as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/08557250442eebc7c86235cdf682b1a9/cf1/bec5fc19800e451dbe3124fb24f21c54 2023-05-10 04:54:07,967 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for b9ae488cd59742d9bf602451afc29300 2023-05-10 04:54:07,967 DEBUG [StoreOpener-e1dcbf091d2db6dbef8d701eb3d89914-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e1dcbf091d2db6dbef8d701eb3d89914/cf1/testSplitTableRegionDeletedRowsDaughter=e0a753dfc7b1edf68920ae1543edc38c-b9ae488cd59742d9bf602451afc29300 2023-05-10 04:54:07,971 INFO [StoreOpener-e1dcbf091d2db6dbef8d701eb3d89914-1] regionserver.HStore(310): Store=e1dcbf091d2db6dbef8d701eb3d89914/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-10 04:54:07,971 INFO [StoreOpener-e1dcbf091d2db6dbef8d701eb3d89914-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region e1dcbf091d2db6dbef8d701eb3d89914 2023-05-10 04:54:07,971 DEBUG [StoreOpener-e1dcbf091d2db6dbef8d701eb3d89914-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e1dcbf091d2db6dbef8d701eb3d89914/cf2 2023-05-10 04:54:07,972 DEBUG [StoreOpener-e1dcbf091d2db6dbef8d701eb3d89914-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e1dcbf091d2db6dbef8d701eb3d89914/cf2 2023-05-10 04:54:07,972 INFO [StoreOpener-e1dcbf091d2db6dbef8d701eb3d89914-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e1dcbf091d2db6dbef8d701eb3d89914 columnFamilyName cf2 2023-05-10 04:54:07,973 INFO [RS:1;jenkins-hbase4:37995-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 08557250442eebc7c86235cdf682b1a9/cf2 of 08557250442eebc7c86235cdf682b1a9 into 09853ed488394ed1906541ea36b30999(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-05-10 04:54:07,973 INFO [RS:1;jenkins-hbase4:37995-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 08557250442eebc7c86235cdf682b1a9/cf1 of 08557250442eebc7c86235cdf682b1a9 into bec5fc19800e451dbe3124fb24f21c54(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-05-10 04:54:07,973 DEBUG [RS:1;jenkins-hbase4:37995-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 08557250442eebc7c86235cdf682b1a9: 2023-05-10 04:54:07,973 DEBUG [RS:1;jenkins-hbase4:37995-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 08557250442eebc7c86235cdf682b1a9: 2023-05-10 04:54:07,973 INFO [RS:1;jenkins-hbase4:37995-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,,1683694447035.08557250442eebc7c86235cdf682b1a9., storeName=08557250442eebc7c86235cdf682b1a9/cf2, priority=9, startTime=1683694447910; duration=0sec 2023-05-10 04:54:07,973 INFO [RS:1;jenkins-hbase4:37995-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,,1683694447035.08557250442eebc7c86235cdf682b1a9., storeName=08557250442eebc7c86235cdf682b1a9/cf1, priority=9, startTime=1683694447910; duration=0sec 2023-05-10 04:54:07,973 DEBUG [RS:1;jenkins-hbase4:37995-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-10 04:54:07,973 DEBUG [RS:1;jenkins-hbase4:37995-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-10 04:54:07,981 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8335cbbc49cf41f1982b7afcfeae3b15.e0a753dfc7b1edf68920ae1543edc38c 2023-05-10 04:54:07,981 DEBUG [StoreOpener-e1dcbf091d2db6dbef8d701eb3d89914-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e1dcbf091d2db6dbef8d701eb3d89914/cf2/8335cbbc49cf41f1982b7afcfeae3b15.e0a753dfc7b1edf68920ae1543edc38c->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf2/8335cbbc49cf41f1982b7afcfeae3b15-top 2023-05-10 04:54:07,984 DEBUG [StoreOpener-e1dcbf091d2db6dbef8d701eb3d89914-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e1dcbf091d2db6dbef8d701eb3d89914/cf2/8dc8ff2e94224e0bb3ca246e89c65dad.e0a753dfc7b1edf68920ae1543edc38c->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf2/8dc8ff2e94224e0bb3ca246e89c65dad-top 2023-05-10 04:54:07,987 DEBUG [StoreOpener-e1dcbf091d2db6dbef8d701eb3d89914-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e1dcbf091d2db6dbef8d701eb3d89914/cf2/bda778111963494983a9d36e39fdf23e.e0a753dfc7b1edf68920ae1543edc38c->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf2/bda778111963494983a9d36e39fdf23e-top 2023-05-10 04:54:07,995 DEBUG [StoreOpener-e1dcbf091d2db6dbef8d701eb3d89914-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e1dcbf091d2db6dbef8d701eb3d89914/cf2/c0c30c5caa4343a39ca186c265342dc5.e0a753dfc7b1edf68920ae1543edc38c->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf2/c0c30c5caa4343a39ca186c265342dc5-top 2023-05-10 04:54:07,998 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 57715c44583d443d9ee24a31f2e0c753 2023-05-10 04:54:07,998 DEBUG [StoreOpener-e1dcbf091d2db6dbef8d701eb3d89914-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e1dcbf091d2db6dbef8d701eb3d89914/cf2/testSplitTableRegionDeletedRowsDaughter=e0a753dfc7b1edf68920ae1543edc38c-57715c44583d443d9ee24a31f2e0c753 2023-05-10 04:54:08,001 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8c9cb9ffa4ef41de964cc06282779ac0 2023-05-10 04:54:08,001 DEBUG [StoreOpener-e1dcbf091d2db6dbef8d701eb3d89914-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e1dcbf091d2db6dbef8d701eb3d89914/cf2/testSplitTableRegionDeletedRowsDaughter=e0a753dfc7b1edf68920ae1543edc38c-8c9cb9ffa4ef41de964cc06282779ac0 2023-05-10 04:54:08,008 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for f70a46cf983c4f61bd80c8814f0701eb 2023-05-10 04:54:08,008 DEBUG [StoreOpener-e1dcbf091d2db6dbef8d701eb3d89914-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e1dcbf091d2db6dbef8d701eb3d89914/cf2/testSplitTableRegionDeletedRowsDaughter=e0a753dfc7b1edf68920ae1543edc38c-f70a46cf983c4f61bd80c8814f0701eb 2023-05-10 04:54:08,008 INFO [StoreOpener-e1dcbf091d2db6dbef8d701eb3d89914-1] regionserver.HStore(310): Store=e1dcbf091d2db6dbef8d701eb3d89914/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-10 04:54:08,009 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e1dcbf091d2db6dbef8d701eb3d89914 2023-05-10 04:54:08,010 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e1dcbf091d2db6dbef8d701eb3d89914 2023-05-10 04:54:08,011 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionDeletedRowsDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-05-10 04:54:08,012 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for e1dcbf091d2db6dbef8d701eb3d89914 2023-05-10 04:54:08,013 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened e1dcbf091d2db6dbef8d701eb3d89914; next sequenceid=107; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60642720, jitterRate=-0.09635305404663086}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-05-10 04:54:08,013 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for e1dcbf091d2db6dbef8d701eb3d89914: 2023-05-10 04:54:08,013 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2334): Post open deploy tasks for testSplitTableRegionDeletedRowsDaughter,60,1683694447035.e1dcbf091d2db6dbef8d701eb3d89914., pid=78, masterSystemTime=1683694447833 2023-05-10 04:54:08,014 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-10 04:54:08,015 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-05-10 04:54:08,015 DEBUG [RS:1;jenkins-hbase4:37995-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-05-10 04:54:08,015 DEBUG [RS:1;jenkins-hbase4:37995-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-05-10 04:54:08,015 INFO [RS:1;jenkins-hbase4:37995-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,60,1683694447035.e1dcbf091d2db6dbef8d701eb3d89914. 2023-05-10 04:54:08,015 INFO [RS:1;jenkins-hbase4:37995-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,60,1683694447035.e1dcbf091d2db6dbef8d701eb3d89914. 2023-05-10 04:54:08,015 DEBUG [RS:1;jenkins-hbase4:37995-shortCompactions-0] regionserver.HStore(1912): e1dcbf091d2db6dbef8d701eb3d89914/cf1 is initiating minor compaction (all files) 2023-05-10 04:54:08,015 DEBUG [RS:1;jenkins-hbase4:37995-longCompactions-0] regionserver.HStore(1912): e1dcbf091d2db6dbef8d701eb3d89914/cf2 is initiating minor compaction (all files) 2023-05-10 04:54:08,015 INFO [RS:1;jenkins-hbase4:37995-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of e1dcbf091d2db6dbef8d701eb3d89914/cf1 in testSplitTableRegionDeletedRowsDaughter,60,1683694447035.e1dcbf091d2db6dbef8d701eb3d89914. 2023-05-10 04:54:08,015 INFO [RS:1;jenkins-hbase4:37995-longCompactions-0] regionserver.HRegion(2259): Starting compaction of e1dcbf091d2db6dbef8d701eb3d89914/cf2 in testSplitTableRegionDeletedRowsDaughter,60,1683694447035.e1dcbf091d2db6dbef8d701eb3d89914. 2023-05-10 04:54:08,015 INFO [RS:1;jenkins-hbase4:37995-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e1dcbf091d2db6dbef8d701eb3d89914/cf2/8dc8ff2e94224e0bb3ca246e89c65dad.e0a753dfc7b1edf68920ae1543edc38c->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf2/8dc8ff2e94224e0bb3ca246e89c65dad-top, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e1dcbf091d2db6dbef8d701eb3d89914/cf2/bda778111963494983a9d36e39fdf23e.e0a753dfc7b1edf68920ae1543edc38c->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf2/bda778111963494983a9d36e39fdf23e-top, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e1dcbf091d2db6dbef8d701eb3d89914/cf2/c0c30c5caa4343a39ca186c265342dc5.e0a753dfc7b1edf68920ae1543edc38c->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf2/c0c30c5caa4343a39ca186c265342dc5-top, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e1dcbf091d2db6dbef8d701eb3d89914/cf2/8335cbbc49cf41f1982b7afcfeae3b15.e0a753dfc7b1edf68920ae1543edc38c->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf2/8335cbbc49cf41f1982b7afcfeae3b15-top, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e1dcbf091d2db6dbef8d701eb3d89914/cf2/testSplitTableRegionDeletedRowsDaughter=e0a753dfc7b1edf68920ae1543edc38c-f70a46cf983c4f61bd80c8814f0701eb, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e1dcbf091d2db6dbef8d701eb3d89914/cf2/testSplitTableRegionDeletedRowsDaughter=e0a753dfc7b1edf68920ae1543edc38c-8c9cb9ffa4ef41de964cc06282779ac0, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e1dcbf091d2db6dbef8d701eb3d89914/cf2/testSplitTableRegionDeletedRowsDaughter=e0a753dfc7b1edf68920ae1543edc38c-57715c44583d443d9ee24a31f2e0c753] into tmpdir=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e1dcbf091d2db6dbef8d701eb3d89914/.tmp, totalSize=34.8 K 2023-05-10 04:54:08,015 INFO [RS:1;jenkins-hbase4:37995-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e1dcbf091d2db6dbef8d701eb3d89914/cf1/9c5dc820063c4ac1b94e51a619cf0ca3.e0a753dfc7b1edf68920ae1543edc38c->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf1/9c5dc820063c4ac1b94e51a619cf0ca3-top, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e1dcbf091d2db6dbef8d701eb3d89914/cf1/5c92186c363e4e2ea144b06cf2ef4148.e0a753dfc7b1edf68920ae1543edc38c->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf1/5c92186c363e4e2ea144b06cf2ef4148-top, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e1dcbf091d2db6dbef8d701eb3d89914/cf1/4391ef9079694b3d9c4aa820517a211f.e0a753dfc7b1edf68920ae1543edc38c->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf1/4391ef9079694b3d9c4aa820517a211f-top, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e1dcbf091d2db6dbef8d701eb3d89914/cf1/6fcee43348954f60b12fefd67f843fe5.e0a753dfc7b1edf68920ae1543edc38c->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf1/6fcee43348954f60b12fefd67f843fe5-top, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e1dcbf091d2db6dbef8d701eb3d89914/cf1/testSplitTableRegionDeletedRowsDaughter=e0a753dfc7b1edf68920ae1543edc38c-b9ae488cd59742d9bf602451afc29300, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e1dcbf091d2db6dbef8d701eb3d89914/cf1/testSplitTableRegionDeletedRowsDaughter=e0a753dfc7b1edf68920ae1543edc38c-4916dab2636c42c190c9d9d879485e00, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e1dcbf091d2db6dbef8d701eb3d89914/cf1/testSplitTableRegionDeletedRowsDaughter=e0a753dfc7b1edf68920ae1543edc38c-aab44d0818ef4a9791980635a3fffd5d] into tmpdir=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e1dcbf091d2db6dbef8d701eb3d89914/.tmp, totalSize=34.8 K 2023-05-10 04:54:08,016 DEBUG [RS:1;jenkins-hbase4:37995-longCompactions-0] compactions.Compactor(207): Compacting 8dc8ff2e94224e0bb3ca246e89c65dad.e0a753dfc7b1edf68920ae1543edc38c, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1683694445340 2023-05-10 04:54:08,016 DEBUG [RS:1;jenkins-hbase4:37995-shortCompactions-0] compactions.Compactor(207): Compacting 9c5dc820063c4ac1b94e51a619cf0ca3.e0a753dfc7b1edf68920ae1543edc38c, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1683694445340 2023-05-10 04:54:08,016 DEBUG [RS:1;jenkins-hbase4:37995-longCompactions-0] compactions.Compactor(207): Compacting bda778111963494983a9d36e39fdf23e.e0a753dfc7b1edf68920ae1543edc38c, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1683694445540 2023-05-10 04:54:08,016 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2361): Finished post open deploy task for testSplitTableRegionDeletedRowsDaughter,60,1683694447035.e1dcbf091d2db6dbef8d701eb3d89914. 2023-05-10 04:54:08,016 DEBUG [RS:1;jenkins-hbase4:37995-shortCompactions-0] compactions.Compactor(207): Compacting 5c92186c363e4e2ea144b06cf2ef4148.e0a753dfc7b1edf68920ae1543edc38c, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1683694445540 2023-05-10 04:54:08,017 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionDeletedRowsDaughter,60,1683694447035.e1dcbf091d2db6dbef8d701eb3d89914. 2023-05-10 04:54:08,017 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=e1dcbf091d2db6dbef8d701eb3d89914, regionState=OPEN, openSeqNum=107, regionLocation=jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:08,017 DEBUG [RS:1;jenkins-hbase4:37995-longCompactions-0] compactions.Compactor(207): Compacting c0c30c5caa4343a39ca186c265342dc5.e0a753dfc7b1edf68920ae1543edc38c, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1683694445737 2023-05-10 04:54:08,017 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionDeletedRowsDaughter,60,1683694447035.e1dcbf091d2db6dbef8d701eb3d89914.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1683694448017"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1683694448017"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1683694448017"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683694448017"}]},"ts":"1683694448017"} 2023-05-10 04:54:08,017 DEBUG [RS:1;jenkins-hbase4:37995-shortCompactions-0] compactions.Compactor(207): Compacting 4391ef9079694b3d9c4aa820517a211f.e0a753dfc7b1edf68920ae1543edc38c, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1683694445737 2023-05-10 04:54:08,018 DEBUG [RS:1;jenkins-hbase4:37995-longCompactions-0] compactions.Compactor(207): Compacting 8335cbbc49cf41f1982b7afcfeae3b15.e0a753dfc7b1edf68920ae1543edc38c, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=84, earliestPutTs=1683694446490 2023-05-10 04:54:08,018 DEBUG [RS:1;jenkins-hbase4:37995-shortCompactions-0] compactions.Compactor(207): Compacting 6fcee43348954f60b12fefd67f843fe5.e0a753dfc7b1edf68920ae1543edc38c, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=84, earliestPutTs=1683694446490 2023-05-10 04:54:08,018 DEBUG [RS:1;jenkins-hbase4:37995-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=e0a753dfc7b1edf68920ae1543edc38c-f70a46cf983c4f61bd80c8814f0701eb, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=91, earliestPutTs=9223372036854775807 2023-05-10 04:54:08,018 DEBUG [RS:1;jenkins-hbase4:37995-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=e0a753dfc7b1edf68920ae1543edc38c-b9ae488cd59742d9bf602451afc29300, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=91, earliestPutTs=9223372036854775807 2023-05-10 04:54:08,018 DEBUG [RS:1;jenkins-hbase4:37995-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=e0a753dfc7b1edf68920ae1543edc38c-8c9cb9ffa4ef41de964cc06282779ac0, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=99, earliestPutTs=9223372036854775807 2023-05-10 04:54:08,019 DEBUG [RS:1;jenkins-hbase4:37995-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=e0a753dfc7b1edf68920ae1543edc38c-4916dab2636c42c190c9d9d879485e00, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=99, earliestPutTs=9223372036854775807 2023-05-10 04:54:08,019 DEBUG [RS:1;jenkins-hbase4:37995-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=e0a753dfc7b1edf68920ae1543edc38c-57715c44583d443d9ee24a31f2e0c753, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-05-10 04:54:08,019 DEBUG [RS:1;jenkins-hbase4:37995-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=e0a753dfc7b1edf68920ae1543edc38c-aab44d0818ef4a9791980635a3fffd5d, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-05-10 04:54:08,020 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=78, resume processing ppid=76 2023-05-10 04:54:08,020 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=78, ppid=76, state=SUCCESS; OpenRegionProcedure e1dcbf091d2db6dbef8d701eb3d89914, server=jenkins-hbase4.apache.org,37995,1683694419499 in 332 msec 2023-05-10 04:54:08,022 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=76, resume processing ppid=72 2023-05-10 04:54:08,022 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=76, ppid=72, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=e1dcbf091d2db6dbef8d701eb3d89914, ASSIGN in 494 msec 2023-05-10 04:54:08,023 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=e0a753dfc7b1edf68920ae1543edc38c, daughterA=08557250442eebc7c86235cdf682b1a9, daughterB=e1dcbf091d2db6dbef8d701eb3d89914 in 987 msec 2023-05-10 04:54:08,034 INFO [RS:1;jenkins-hbase4:37995-longCompactions-0] throttle.PressureAwareThroughputController(145): e1dcbf091d2db6dbef8d701eb3d89914#cf2#compaction#75 average throughput is 0.00 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-05-10 04:54:08,034 INFO [RS:1;jenkins-hbase4:37995-shortCompactions-0] throttle.PressureAwareThroughputController(145): e1dcbf091d2db6dbef8d701eb3d89914#cf1#compaction#74 average throughput is 0.00 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-05-10 04:54:08,056 DEBUG [Listener at localhost/45007] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-05-10 04:54:08,062 INFO [RS-EventLoopGroup-4-2] ipc.ServerRpcConnection(540): Connection from 172.31.14.131:47096, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-05-10 04:54:08,063 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37995] regionserver.RSRpcServices(1662): Compacting testSplitTableRegionDeletedRowsDaughter,,1683694447035.08557250442eebc7c86235cdf682b1a9. 2023-05-10 04:54:08,064 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37995] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-05-10 04:54:08,064 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37995] regionserver.HStore(1912): 08557250442eebc7c86235cdf682b1a9/cf1 is initiating major compaction (all files) 2023-05-10 04:54:08,064 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37995] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@78b2449f; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,,1683694447035.08557250442eebc7c86235cdf682b1a9.; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-05-10 04:54:08,064 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37995] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-05-10 04:54:08,064 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37995] regionserver.HStore(1912): 08557250442eebc7c86235cdf682b1a9/cf2 is initiating major compaction (all files) 2023-05-10 04:54:08,064 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37995] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@452298d; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,,1683694447035.08557250442eebc7c86235cdf682b1a9.; compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-05-10 04:54:08,066 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37995] regionserver.RSRpcServices(1662): Compacting testSplitTableRegionDeletedRowsDaughter,60,1683694447035.e1dcbf091d2db6dbef8d701eb3d89914. 2023-05-10 04:54:08,066 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37995] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 7 compacting, 0 eligible, 16 blocking 2023-05-10 04:54:08,066 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37995] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 0 files of size 0 starting at candidate #-1 after considering 0 permutations with 0 in ratio 2023-05-10 04:54:08,066 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37995] compactions.SortedCompactionPolicy(232): Not compacting files because we only have 0 files ready for compaction. Need 3 to initiate. 2023-05-10 04:54:08,066 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37995] regionserver.CompactSplit(415): Not compacting testSplitTableRegionDeletedRowsDaughter,60,1683694447035.e1dcbf091d2db6dbef8d701eb3d89914. because compaction request was cancelled 2023-05-10 04:54:08,066 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37995] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 7 compacting, 0 eligible, 16 blocking 2023-05-10 04:54:08,066 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37995] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 0 files of size 0 starting at candidate #-1 after considering 0 permutations with 0 in ratio 2023-05-10 04:54:08,067 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37995] compactions.SortedCompactionPolicy(232): Not compacting files because we only have 0 files ready for compaction. Need 3 to initiate. 2023-05-10 04:54:08,067 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37995] regionserver.CompactSplit(415): Not compacting testSplitTableRegionDeletedRowsDaughter,60,1683694447035.e1dcbf091d2db6dbef8d701eb3d89914. because compaction request was cancelled 2023-05-10 04:54:08,067 INFO [Listener at localhost/45007] hbase.Waiter(180): Waiting up to [6,000] milli-secs(wait.for.ratio=[1]) 2023-05-10 04:54:08,078 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37995] regionserver.HRegion(1451): Region testSplitTableRegionDeletedRowsDaughter,,1683694447035.08557250442eebc7c86235cdf682b1a9. is not mergeable because it has references 2023-05-10 04:54:08,078 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37995] regionserver.HRegion(1451): Region testSplitTableRegionDeletedRowsDaughter,60,1683694447035.e1dcbf091d2db6dbef8d701eb3d89914. is not mergeable because it has references 2023-05-10 04:54:08,186 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37995] regionserver.HRegion(1451): Region testSplitTableRegionDeletedRowsDaughter,,1683694447035.08557250442eebc7c86235cdf682b1a9. is not mergeable because it has references 2023-05-10 04:54:08,187 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37995] regionserver.HRegion(1451): Region testSplitTableRegionDeletedRowsDaughter,60,1683694447035.e1dcbf091d2db6dbef8d701eb3d89914. is not mergeable because it has references 2023-05-10 04:54:08,295 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37995] regionserver.HRegion(1451): Region testSplitTableRegionDeletedRowsDaughter,,1683694447035.08557250442eebc7c86235cdf682b1a9. is not mergeable because it has references 2023-05-10 04:54:08,296 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37995] regionserver.HRegion(1451): Region testSplitTableRegionDeletedRowsDaughter,60,1683694447035.e1dcbf091d2db6dbef8d701eb3d89914. is not mergeable because it has references 2023-05-10 04:54:08,404 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37995] regionserver.HRegion(1451): Region testSplitTableRegionDeletedRowsDaughter,,1683694447035.08557250442eebc7c86235cdf682b1a9. is not mergeable because it has references 2023-05-10 04:54:08,404 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37995] regionserver.HRegion(1451): Region testSplitTableRegionDeletedRowsDaughter,60,1683694447035.e1dcbf091d2db6dbef8d701eb3d89914. is not mergeable because it has references 2023-05-10 04:54:08,456 DEBUG [RS:1;jenkins-hbase4:37995-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 21c6268577fd48f6b60d6f4a64b0df16: NONE, but ROW specified in column family configuration 2023-05-10 04:54:08,456 DEBUG [RS:1;jenkins-hbase4:37995-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 49afb61cf8544b59ac7da765b1018b1c: NONE, but ROW specified in column family configuration 2023-05-10 04:54:08,457 DEBUG [RS:1;jenkins-hbase4:37995-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e1dcbf091d2db6dbef8d701eb3d89914/.tmp/cf1/21c6268577fd48f6b60d6f4a64b0df16 as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e1dcbf091d2db6dbef8d701eb3d89914/cf1/21c6268577fd48f6b60d6f4a64b0df16 2023-05-10 04:54:08,457 DEBUG [RS:1;jenkins-hbase4:37995-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e1dcbf091d2db6dbef8d701eb3d89914/.tmp/cf2/49afb61cf8544b59ac7da765b1018b1c as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e1dcbf091d2db6dbef8d701eb3d89914/cf2/49afb61cf8544b59ac7da765b1018b1c 2023-05-10 04:54:08,460 DEBUG [RS:1;jenkins-hbase4:37995-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 49afb61cf8544b59ac7da765b1018b1c: NONE, but ROW specified in column family configuration 2023-05-10 04:54:08,460 DEBUG [RS:1;jenkins-hbase4:37995-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 21c6268577fd48f6b60d6f4a64b0df16: NONE, but ROW specified in column family configuration 2023-05-10 04:54:08,462 INFO [RS:1;jenkins-hbase4:37995-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in e1dcbf091d2db6dbef8d701eb3d89914/cf1 of e1dcbf091d2db6dbef8d701eb3d89914 into 21c6268577fd48f6b60d6f4a64b0df16(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-05-10 04:54:08,462 INFO [RS:1;jenkins-hbase4:37995-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in e1dcbf091d2db6dbef8d701eb3d89914/cf2 of e1dcbf091d2db6dbef8d701eb3d89914 into 49afb61cf8544b59ac7da765b1018b1c(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-05-10 04:54:08,462 DEBUG [RS:1;jenkins-hbase4:37995-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for e1dcbf091d2db6dbef8d701eb3d89914: 2023-05-10 04:54:08,462 DEBUG [RS:1;jenkins-hbase4:37995-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for e1dcbf091d2db6dbef8d701eb3d89914: 2023-05-10 04:54:08,462 INFO [RS:1;jenkins-hbase4:37995-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,60,1683694447035.e1dcbf091d2db6dbef8d701eb3d89914., storeName=e1dcbf091d2db6dbef8d701eb3d89914/cf1, priority=9, startTime=1683694448014; duration=0sec 2023-05-10 04:54:08,462 INFO [RS:1;jenkins-hbase4:37995-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,60,1683694447035.e1dcbf091d2db6dbef8d701eb3d89914., storeName=e1dcbf091d2db6dbef8d701eb3d89914/cf2, priority=9, startTime=1683694448015; duration=0sec 2023-05-10 04:54:08,462 DEBUG [RS:1;jenkins-hbase4:37995-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-05-10 04:54:08,462 DEBUG [RS:1;jenkins-hbase4:37995-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-05-10 04:54:08,462 INFO [RS:1;jenkins-hbase4:37995-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 08557250442eebc7c86235cdf682b1a9/cf2 in testSplitTableRegionDeletedRowsDaughter,,1683694447035.08557250442eebc7c86235cdf682b1a9. 2023-05-10 04:54:08,462 INFO [RS:1;jenkins-hbase4:37995-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 08557250442eebc7c86235cdf682b1a9/cf1 in testSplitTableRegionDeletedRowsDaughter,,1683694447035.08557250442eebc7c86235cdf682b1a9. 2023-05-10 04:54:08,462 INFO [RS:1;jenkins-hbase4:37995-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/08557250442eebc7c86235cdf682b1a9/cf2/09853ed488394ed1906541ea36b30999] into tmpdir=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/08557250442eebc7c86235cdf682b1a9/.tmp, totalSize=6.8 K 2023-05-10 04:54:08,462 INFO [RS:1;jenkins-hbase4:37995-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/08557250442eebc7c86235cdf682b1a9/cf1/bec5fc19800e451dbe3124fb24f21c54] into tmpdir=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/08557250442eebc7c86235cdf682b1a9/.tmp, totalSize=6.8 K 2023-05-10 04:54:08,463 DEBUG [RS:1;jenkins-hbase4:37995-shortCompactions-0] compactions.Compactor(207): Compacting 09853ed488394ed1906541ea36b30999, keycount=49, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1683694445340 2023-05-10 04:54:08,463 DEBUG [RS:1;jenkins-hbase4:37995-longCompactions-0] compactions.Compactor(207): Compacting bec5fc19800e451dbe3124fb24f21c54, keycount=49, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1683694445340 2023-05-10 04:54:08,467 INFO [RS:1;jenkins-hbase4:37995-shortCompactions-0] throttle.PressureAwareThroughputController(145): 08557250442eebc7c86235cdf682b1a9#cf2#compaction#76 average throughput is 1.40 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-05-10 04:54:08,467 INFO [RS:1;jenkins-hbase4:37995-longCompactions-0] throttle.PressureAwareThroughputController(145): 08557250442eebc7c86235cdf682b1a9#cf1#compaction#77 average throughput is 1.40 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-05-10 04:54:08,482 DEBUG [RS:1;jenkins-hbase4:37995-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/08557250442eebc7c86235cdf682b1a9/.tmp/cf2/de25529ed5d44f04be9f70f3fc3deb08 as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/08557250442eebc7c86235cdf682b1a9/cf2/de25529ed5d44f04be9f70f3fc3deb08 2023-05-10 04:54:08,483 DEBUG [RS:1;jenkins-hbase4:37995-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/08557250442eebc7c86235cdf682b1a9/.tmp/cf1/bc79127118d74a76a5be90840e50d848 as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/08557250442eebc7c86235cdf682b1a9/cf1/bc79127118d74a76a5be90840e50d848 2023-05-10 04:54:08,488 INFO [RS:1;jenkins-hbase4:37995-shortCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in 08557250442eebc7c86235cdf682b1a9/cf2 of 08557250442eebc7c86235cdf682b1a9 into de25529ed5d44f04be9f70f3fc3deb08(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-05-10 04:54:08,488 INFO [RS:1;jenkins-hbase4:37995-longCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in 08557250442eebc7c86235cdf682b1a9/cf1 of 08557250442eebc7c86235cdf682b1a9 into bc79127118d74a76a5be90840e50d848(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-05-10 04:54:08,488 DEBUG [RS:1;jenkins-hbase4:37995-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 08557250442eebc7c86235cdf682b1a9: 2023-05-10 04:54:08,488 DEBUG [RS:1;jenkins-hbase4:37995-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 08557250442eebc7c86235cdf682b1a9: 2023-05-10 04:54:08,490 INFO [RS:1;jenkins-hbase4:37995-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,,1683694447035.08557250442eebc7c86235cdf682b1a9., storeName=cf1, fileCount=1, fileSize=6.8 K (6.8 K), priority=1, time=1683694448064; duration=0sec 2023-05-10 04:54:08,490 INFO [RS:1;jenkins-hbase4:37995-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,,1683694447035.08557250442eebc7c86235cdf682b1a9., storeName=cf2, fileCount=1, fileSize=6.8 K (6.8 K), priority=1, time=1683694448064; duration=0sec 2023-05-10 04:54:08,490 DEBUG [RS:1;jenkins-hbase4:37995-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-10 04:54:08,490 DEBUG [RS:1;jenkins-hbase4:37995-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-10 04:54:08,512 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=37995] regionserver.HRegion(1451): Region testSplitTableRegionDeletedRowsDaughter,,1683694447035.08557250442eebc7c86235cdf682b1a9. is not mergeable because it has references 2023-05-10 04:54:08,512 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37995] regionserver.HRegion(1451): Region testSplitTableRegionDeletedRowsDaughter,60,1683694447035.e1dcbf091d2db6dbef8d701eb3d89914. is not mergeable because it has references 2023-05-10 04:54:08,518 DEBUG [Listener at localhost/45007] client.ClientScanner(240): Advancing internal scanner to startKey at '60', inclusive 2023-05-10 04:54:08,519 WARN [Listener at localhost/45007] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-05-10 04:54:08,521 INFO [Listener at localhost/45007] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:08,521 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:08,522 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] procedure2.ProcedureExecutor(1029): Stored pid=79, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:08,524 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-05-10 04:54:08,525 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683694448524"}]},"ts":"1683694448524"} 2023-05-10 04:54:08,526 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=DISABLING in hbase:meta 2023-05-10 04:54:08,527 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionDeletedRowsDaughter to state=DISABLING 2023-05-10 04:54:08,528 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=08557250442eebc7c86235cdf682b1a9, UNASSIGN}, {pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=e1dcbf091d2db6dbef8d701eb3d89914, UNASSIGN}] 2023-05-10 04:54:08,528 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=e1dcbf091d2db6dbef8d701eb3d89914, UNASSIGN 2023-05-10 04:54:08,529 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=e1dcbf091d2db6dbef8d701eb3d89914, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:08,529 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,60,1683694447035.e1dcbf091d2db6dbef8d701eb3d89914.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1683694448529"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683694448529"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683694448529"}]},"ts":"1683694448529"} 2023-05-10 04:54:08,530 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=82, ppid=81, state=RUNNABLE; CloseRegionProcedure e1dcbf091d2db6dbef8d701eb3d89914, server=jenkins-hbase4.apache.org,37995,1683694419499}] 2023-05-10 04:54:08,532 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=08557250442eebc7c86235cdf682b1a9, UNASSIGN 2023-05-10 04:54:08,533 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=08557250442eebc7c86235cdf682b1a9, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:08,533 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1683694447035.08557250442eebc7c86235cdf682b1a9.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1683694448533"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683694448533"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683694448533"}]},"ts":"1683694448533"} 2023-05-10 04:54:08,534 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=83, ppid=80, state=RUNNABLE; CloseRegionProcedure 08557250442eebc7c86235cdf682b1a9, server=jenkins-hbase4.apache.org,37995,1683694419499}] 2023-05-10 04:54:08,625 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-05-10 04:54:08,682 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 08557250442eebc7c86235cdf682b1a9 2023-05-10 04:54:08,682 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 08557250442eebc7c86235cdf682b1a9, disabling compactions & flushes 2023-05-10 04:54:08,682 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionDeletedRowsDaughter,,1683694447035.08557250442eebc7c86235cdf682b1a9. 2023-05-10 04:54:08,682 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,,1683694447035.08557250442eebc7c86235cdf682b1a9. 2023-05-10 04:54:08,682 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,,1683694447035.08557250442eebc7c86235cdf682b1a9. after waiting 0 ms 2023-05-10 04:54:08,682 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,,1683694447035.08557250442eebc7c86235cdf682b1a9. 2023-05-10 04:54:08,683 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1683694447035.08557250442eebc7c86235cdf682b1a9.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/08557250442eebc7c86235cdf682b1a9/cf1/9c5dc820063c4ac1b94e51a619cf0ca3.e0a753dfc7b1edf68920ae1543edc38c->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf1/9c5dc820063c4ac1b94e51a619cf0ca3-bottom, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/08557250442eebc7c86235cdf682b1a9/cf1/5c92186c363e4e2ea144b06cf2ef4148.e0a753dfc7b1edf68920ae1543edc38c->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf1/5c92186c363e4e2ea144b06cf2ef4148-bottom, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/08557250442eebc7c86235cdf682b1a9/cf1/4391ef9079694b3d9c4aa820517a211f.e0a753dfc7b1edf68920ae1543edc38c->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf1/4391ef9079694b3d9c4aa820517a211f-bottom, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/08557250442eebc7c86235cdf682b1a9/cf1/testSplitTableRegionDeletedRowsDaughter=e0a753dfc7b1edf68920ae1543edc38c-bee3c0539163404a9c5862a802d06fc8, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/08557250442eebc7c86235cdf682b1a9/cf1/testSplitTableRegionDeletedRowsDaughter=e0a753dfc7b1edf68920ae1543edc38c-6abf15e207cc4ef7bb40e895775d046c, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/08557250442eebc7c86235cdf682b1a9/cf1/testSplitTableRegionDeletedRowsDaughter=e0a753dfc7b1edf68920ae1543edc38c-069e6767796b4374972ea4f62b4268aa, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/08557250442eebc7c86235cdf682b1a9/cf1/bec5fc19800e451dbe3124fb24f21c54, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/08557250442eebc7c86235cdf682b1a9/cf1/6fcee43348954f60b12fefd67f843fe5.e0a753dfc7b1edf68920ae1543edc38c->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf1/6fcee43348954f60b12fefd67f843fe5-bottom] to archive 2023-05-10 04:54:08,684 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1683694447035.08557250442eebc7c86235cdf682b1a9.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-05-10 04:54:08,685 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1683694447035.08557250442eebc7c86235cdf682b1a9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/08557250442eebc7c86235cdf682b1a9/cf1/9c5dc820063c4ac1b94e51a619cf0ca3.e0a753dfc7b1edf68920ae1543edc38c to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/08557250442eebc7c86235cdf682b1a9/cf1/9c5dc820063c4ac1b94e51a619cf0ca3.e0a753dfc7b1edf68920ae1543edc38c 2023-05-10 04:54:08,686 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1683694447035.08557250442eebc7c86235cdf682b1a9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/08557250442eebc7c86235cdf682b1a9/cf1/5c92186c363e4e2ea144b06cf2ef4148.e0a753dfc7b1edf68920ae1543edc38c to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/08557250442eebc7c86235cdf682b1a9/cf1/5c92186c363e4e2ea144b06cf2ef4148.e0a753dfc7b1edf68920ae1543edc38c 2023-05-10 04:54:08,688 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1683694447035.08557250442eebc7c86235cdf682b1a9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/08557250442eebc7c86235cdf682b1a9/cf1/4391ef9079694b3d9c4aa820517a211f.e0a753dfc7b1edf68920ae1543edc38c to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/08557250442eebc7c86235cdf682b1a9/cf1/4391ef9079694b3d9c4aa820517a211f.e0a753dfc7b1edf68920ae1543edc38c 2023-05-10 04:54:08,689 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1683694447035.08557250442eebc7c86235cdf682b1a9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/08557250442eebc7c86235cdf682b1a9/cf1/testSplitTableRegionDeletedRowsDaughter=e0a753dfc7b1edf68920ae1543edc38c-bee3c0539163404a9c5862a802d06fc8 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/08557250442eebc7c86235cdf682b1a9/cf1/testSplitTableRegionDeletedRowsDaughter=e0a753dfc7b1edf68920ae1543edc38c-bee3c0539163404a9c5862a802d06fc8 2023-05-10 04:54:08,690 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1683694447035.08557250442eebc7c86235cdf682b1a9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/08557250442eebc7c86235cdf682b1a9/cf1/testSplitTableRegionDeletedRowsDaughter=e0a753dfc7b1edf68920ae1543edc38c-6abf15e207cc4ef7bb40e895775d046c to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/08557250442eebc7c86235cdf682b1a9/cf1/testSplitTableRegionDeletedRowsDaughter=e0a753dfc7b1edf68920ae1543edc38c-6abf15e207cc4ef7bb40e895775d046c 2023-05-10 04:54:08,691 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1683694447035.08557250442eebc7c86235cdf682b1a9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/08557250442eebc7c86235cdf682b1a9/cf1/testSplitTableRegionDeletedRowsDaughter=e0a753dfc7b1edf68920ae1543edc38c-069e6767796b4374972ea4f62b4268aa to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/08557250442eebc7c86235cdf682b1a9/cf1/testSplitTableRegionDeletedRowsDaughter=e0a753dfc7b1edf68920ae1543edc38c-069e6767796b4374972ea4f62b4268aa 2023-05-10 04:54:08,692 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1683694447035.08557250442eebc7c86235cdf682b1a9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/08557250442eebc7c86235cdf682b1a9/cf1/bec5fc19800e451dbe3124fb24f21c54 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/08557250442eebc7c86235cdf682b1a9/cf1/bec5fc19800e451dbe3124fb24f21c54 2023-05-10 04:54:08,693 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1683694447035.08557250442eebc7c86235cdf682b1a9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/08557250442eebc7c86235cdf682b1a9/cf1/6fcee43348954f60b12fefd67f843fe5.e0a753dfc7b1edf68920ae1543edc38c to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/08557250442eebc7c86235cdf682b1a9/cf1/6fcee43348954f60b12fefd67f843fe5.e0a753dfc7b1edf68920ae1543edc38c 2023-05-10 04:54:08,695 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1683694447035.08557250442eebc7c86235cdf682b1a9.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/08557250442eebc7c86235cdf682b1a9/cf2/8dc8ff2e94224e0bb3ca246e89c65dad.e0a753dfc7b1edf68920ae1543edc38c->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf2/8dc8ff2e94224e0bb3ca246e89c65dad-bottom, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/08557250442eebc7c86235cdf682b1a9/cf2/bda778111963494983a9d36e39fdf23e.e0a753dfc7b1edf68920ae1543edc38c->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf2/bda778111963494983a9d36e39fdf23e-bottom, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/08557250442eebc7c86235cdf682b1a9/cf2/c0c30c5caa4343a39ca186c265342dc5.e0a753dfc7b1edf68920ae1543edc38c->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf2/c0c30c5caa4343a39ca186c265342dc5-bottom, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/08557250442eebc7c86235cdf682b1a9/cf2/testSplitTableRegionDeletedRowsDaughter=e0a753dfc7b1edf68920ae1543edc38c-fcb908e83d9d411482f19b7258cd7021, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/08557250442eebc7c86235cdf682b1a9/cf2/testSplitTableRegionDeletedRowsDaughter=e0a753dfc7b1edf68920ae1543edc38c-839ff9ffc87b4074b7e77f7b6e45b9ce, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/08557250442eebc7c86235cdf682b1a9/cf2/testSplitTableRegionDeletedRowsDaughter=e0a753dfc7b1edf68920ae1543edc38c-e8a131ba8e924d2ab9e7dd5b50323810, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/08557250442eebc7c86235cdf682b1a9/cf2/09853ed488394ed1906541ea36b30999, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/08557250442eebc7c86235cdf682b1a9/cf2/8335cbbc49cf41f1982b7afcfeae3b15.e0a753dfc7b1edf68920ae1543edc38c->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf2/8335cbbc49cf41f1982b7afcfeae3b15-bottom] to archive 2023-05-10 04:54:08,695 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1683694447035.08557250442eebc7c86235cdf682b1a9.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-05-10 04:54:08,697 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1683694447035.08557250442eebc7c86235cdf682b1a9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/08557250442eebc7c86235cdf682b1a9/cf2/8dc8ff2e94224e0bb3ca246e89c65dad.e0a753dfc7b1edf68920ae1543edc38c to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/08557250442eebc7c86235cdf682b1a9/cf2/8dc8ff2e94224e0bb3ca246e89c65dad.e0a753dfc7b1edf68920ae1543edc38c 2023-05-10 04:54:08,698 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1683694447035.08557250442eebc7c86235cdf682b1a9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/08557250442eebc7c86235cdf682b1a9/cf2/bda778111963494983a9d36e39fdf23e.e0a753dfc7b1edf68920ae1543edc38c to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/08557250442eebc7c86235cdf682b1a9/cf2/bda778111963494983a9d36e39fdf23e.e0a753dfc7b1edf68920ae1543edc38c 2023-05-10 04:54:08,698 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1683694447035.08557250442eebc7c86235cdf682b1a9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/08557250442eebc7c86235cdf682b1a9/cf2/c0c30c5caa4343a39ca186c265342dc5.e0a753dfc7b1edf68920ae1543edc38c to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/08557250442eebc7c86235cdf682b1a9/cf2/c0c30c5caa4343a39ca186c265342dc5.e0a753dfc7b1edf68920ae1543edc38c 2023-05-10 04:54:08,699 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1683694447035.08557250442eebc7c86235cdf682b1a9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/08557250442eebc7c86235cdf682b1a9/cf2/testSplitTableRegionDeletedRowsDaughter=e0a753dfc7b1edf68920ae1543edc38c-fcb908e83d9d411482f19b7258cd7021 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/08557250442eebc7c86235cdf682b1a9/cf2/testSplitTableRegionDeletedRowsDaughter=e0a753dfc7b1edf68920ae1543edc38c-fcb908e83d9d411482f19b7258cd7021 2023-05-10 04:54:08,700 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1683694447035.08557250442eebc7c86235cdf682b1a9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/08557250442eebc7c86235cdf682b1a9/cf2/testSplitTableRegionDeletedRowsDaughter=e0a753dfc7b1edf68920ae1543edc38c-839ff9ffc87b4074b7e77f7b6e45b9ce to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/08557250442eebc7c86235cdf682b1a9/cf2/testSplitTableRegionDeletedRowsDaughter=e0a753dfc7b1edf68920ae1543edc38c-839ff9ffc87b4074b7e77f7b6e45b9ce 2023-05-10 04:54:08,701 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1683694447035.08557250442eebc7c86235cdf682b1a9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/08557250442eebc7c86235cdf682b1a9/cf2/testSplitTableRegionDeletedRowsDaughter=e0a753dfc7b1edf68920ae1543edc38c-e8a131ba8e924d2ab9e7dd5b50323810 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/08557250442eebc7c86235cdf682b1a9/cf2/testSplitTableRegionDeletedRowsDaughter=e0a753dfc7b1edf68920ae1543edc38c-e8a131ba8e924d2ab9e7dd5b50323810 2023-05-10 04:54:08,702 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1683694447035.08557250442eebc7c86235cdf682b1a9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/08557250442eebc7c86235cdf682b1a9/cf2/09853ed488394ed1906541ea36b30999 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/08557250442eebc7c86235cdf682b1a9/cf2/09853ed488394ed1906541ea36b30999 2023-05-10 04:54:08,703 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1683694447035.08557250442eebc7c86235cdf682b1a9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/08557250442eebc7c86235cdf682b1a9/cf2/8335cbbc49cf41f1982b7afcfeae3b15.e0a753dfc7b1edf68920ae1543edc38c to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/08557250442eebc7c86235cdf682b1a9/cf2/8335cbbc49cf41f1982b7afcfeae3b15.e0a753dfc7b1edf68920ae1543edc38c 2023-05-10 04:54:08,706 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/08557250442eebc7c86235cdf682b1a9/recovered.edits/113.seqid, newMaxSeqId=113, maxSeqId=106 2023-05-10 04:54:08,706 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-10 04:54:08,706 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionDeletedRowsDaughter,,1683694447035.08557250442eebc7c86235cdf682b1a9. 2023-05-10 04:54:08,706 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 08557250442eebc7c86235cdf682b1a9: 2023-05-10 04:54:08,708 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 08557250442eebc7c86235cdf682b1a9 2023-05-10 04:54:08,708 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close e1dcbf091d2db6dbef8d701eb3d89914 2023-05-10 04:54:08,708 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing e1dcbf091d2db6dbef8d701eb3d89914, disabling compactions & flushes 2023-05-10 04:54:08,708 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionDeletedRowsDaughter,60,1683694447035.e1dcbf091d2db6dbef8d701eb3d89914. 2023-05-10 04:54:08,708 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,60,1683694447035.e1dcbf091d2db6dbef8d701eb3d89914. 2023-05-10 04:54:08,708 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,60,1683694447035.e1dcbf091d2db6dbef8d701eb3d89914. after waiting 0 ms 2023-05-10 04:54:08,708 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,60,1683694447035.e1dcbf091d2db6dbef8d701eb3d89914. 2023-05-10 04:54:08,708 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=08557250442eebc7c86235cdf682b1a9, regionState=CLOSED 2023-05-10 04:54:08,708 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,,1683694447035.08557250442eebc7c86235cdf682b1a9.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1683694448708"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683694448708"}]},"ts":"1683694448708"} 2023-05-10 04:54:08,709 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1683694447035.e1dcbf091d2db6dbef8d701eb3d89914.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e1dcbf091d2db6dbef8d701eb3d89914/cf1/9c5dc820063c4ac1b94e51a619cf0ca3.e0a753dfc7b1edf68920ae1543edc38c->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf1/9c5dc820063c4ac1b94e51a619cf0ca3-top, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e1dcbf091d2db6dbef8d701eb3d89914/cf1/5c92186c363e4e2ea144b06cf2ef4148.e0a753dfc7b1edf68920ae1543edc38c->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf1/5c92186c363e4e2ea144b06cf2ef4148-top, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e1dcbf091d2db6dbef8d701eb3d89914/cf1/4391ef9079694b3d9c4aa820517a211f.e0a753dfc7b1edf68920ae1543edc38c->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf1/4391ef9079694b3d9c4aa820517a211f-top, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e1dcbf091d2db6dbef8d701eb3d89914/cf1/6fcee43348954f60b12fefd67f843fe5.e0a753dfc7b1edf68920ae1543edc38c->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf1/6fcee43348954f60b12fefd67f843fe5-top, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e1dcbf091d2db6dbef8d701eb3d89914/cf1/testSplitTableRegionDeletedRowsDaughter=e0a753dfc7b1edf68920ae1543edc38c-b9ae488cd59742d9bf602451afc29300, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e1dcbf091d2db6dbef8d701eb3d89914/cf1/testSplitTableRegionDeletedRowsDaughter=e0a753dfc7b1edf68920ae1543edc38c-4916dab2636c42c190c9d9d879485e00, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e1dcbf091d2db6dbef8d701eb3d89914/cf1/testSplitTableRegionDeletedRowsDaughter=e0a753dfc7b1edf68920ae1543edc38c-aab44d0818ef4a9791980635a3fffd5d] to archive 2023-05-10 04:54:08,710 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1683694447035.e1dcbf091d2db6dbef8d701eb3d89914.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-05-10 04:54:08,711 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1683694447035.e1dcbf091d2db6dbef8d701eb3d89914.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e1dcbf091d2db6dbef8d701eb3d89914/cf1/9c5dc820063c4ac1b94e51a619cf0ca3.e0a753dfc7b1edf68920ae1543edc38c to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e1dcbf091d2db6dbef8d701eb3d89914/cf1/9c5dc820063c4ac1b94e51a619cf0ca3.e0a753dfc7b1edf68920ae1543edc38c 2023-05-10 04:54:08,711 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=83, resume processing ppid=80 2023-05-10 04:54:08,711 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=83, ppid=80, state=SUCCESS; CloseRegionProcedure 08557250442eebc7c86235cdf682b1a9, server=jenkins-hbase4.apache.org,37995,1683694419499 in 176 msec 2023-05-10 04:54:08,713 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=80, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=08557250442eebc7c86235cdf682b1a9, UNASSIGN in 184 msec 2023-05-10 04:54:08,713 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1683694447035.e1dcbf091d2db6dbef8d701eb3d89914.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e1dcbf091d2db6dbef8d701eb3d89914/cf1/5c92186c363e4e2ea144b06cf2ef4148.e0a753dfc7b1edf68920ae1543edc38c to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e1dcbf091d2db6dbef8d701eb3d89914/cf1/5c92186c363e4e2ea144b06cf2ef4148.e0a753dfc7b1edf68920ae1543edc38c 2023-05-10 04:54:08,714 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1683694447035.e1dcbf091d2db6dbef8d701eb3d89914.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e1dcbf091d2db6dbef8d701eb3d89914/cf1/4391ef9079694b3d9c4aa820517a211f.e0a753dfc7b1edf68920ae1543edc38c to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e1dcbf091d2db6dbef8d701eb3d89914/cf1/4391ef9079694b3d9c4aa820517a211f.e0a753dfc7b1edf68920ae1543edc38c 2023-05-10 04:54:08,715 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1683694447035.e1dcbf091d2db6dbef8d701eb3d89914.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e1dcbf091d2db6dbef8d701eb3d89914/cf1/6fcee43348954f60b12fefd67f843fe5.e0a753dfc7b1edf68920ae1543edc38c to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e1dcbf091d2db6dbef8d701eb3d89914/cf1/6fcee43348954f60b12fefd67f843fe5.e0a753dfc7b1edf68920ae1543edc38c 2023-05-10 04:54:08,716 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1683694447035.e1dcbf091d2db6dbef8d701eb3d89914.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e1dcbf091d2db6dbef8d701eb3d89914/cf1/testSplitTableRegionDeletedRowsDaughter=e0a753dfc7b1edf68920ae1543edc38c-b9ae488cd59742d9bf602451afc29300 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e1dcbf091d2db6dbef8d701eb3d89914/cf1/testSplitTableRegionDeletedRowsDaughter=e0a753dfc7b1edf68920ae1543edc38c-b9ae488cd59742d9bf602451afc29300 2023-05-10 04:54:08,716 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1683694447035.e1dcbf091d2db6dbef8d701eb3d89914.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e1dcbf091d2db6dbef8d701eb3d89914/cf1/testSplitTableRegionDeletedRowsDaughter=e0a753dfc7b1edf68920ae1543edc38c-4916dab2636c42c190c9d9d879485e00 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e1dcbf091d2db6dbef8d701eb3d89914/cf1/testSplitTableRegionDeletedRowsDaughter=e0a753dfc7b1edf68920ae1543edc38c-4916dab2636c42c190c9d9d879485e00 2023-05-10 04:54:08,717 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1683694447035.e1dcbf091d2db6dbef8d701eb3d89914.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e1dcbf091d2db6dbef8d701eb3d89914/cf1/testSplitTableRegionDeletedRowsDaughter=e0a753dfc7b1edf68920ae1543edc38c-aab44d0818ef4a9791980635a3fffd5d to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e1dcbf091d2db6dbef8d701eb3d89914/cf1/testSplitTableRegionDeletedRowsDaughter=e0a753dfc7b1edf68920ae1543edc38c-aab44d0818ef4a9791980635a3fffd5d 2023-05-10 04:54:08,719 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1683694447035.e1dcbf091d2db6dbef8d701eb3d89914.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e1dcbf091d2db6dbef8d701eb3d89914/cf2/8dc8ff2e94224e0bb3ca246e89c65dad.e0a753dfc7b1edf68920ae1543edc38c->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf2/8dc8ff2e94224e0bb3ca246e89c65dad-top, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e1dcbf091d2db6dbef8d701eb3d89914/cf2/bda778111963494983a9d36e39fdf23e.e0a753dfc7b1edf68920ae1543edc38c->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf2/bda778111963494983a9d36e39fdf23e-top, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e1dcbf091d2db6dbef8d701eb3d89914/cf2/c0c30c5caa4343a39ca186c265342dc5.e0a753dfc7b1edf68920ae1543edc38c->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf2/c0c30c5caa4343a39ca186c265342dc5-top, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e1dcbf091d2db6dbef8d701eb3d89914/cf2/8335cbbc49cf41f1982b7afcfeae3b15.e0a753dfc7b1edf68920ae1543edc38c->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf2/8335cbbc49cf41f1982b7afcfeae3b15-top, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e1dcbf091d2db6dbef8d701eb3d89914/cf2/testSplitTableRegionDeletedRowsDaughter=e0a753dfc7b1edf68920ae1543edc38c-f70a46cf983c4f61bd80c8814f0701eb, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e1dcbf091d2db6dbef8d701eb3d89914/cf2/testSplitTableRegionDeletedRowsDaughter=e0a753dfc7b1edf68920ae1543edc38c-8c9cb9ffa4ef41de964cc06282779ac0, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e1dcbf091d2db6dbef8d701eb3d89914/cf2/testSplitTableRegionDeletedRowsDaughter=e0a753dfc7b1edf68920ae1543edc38c-57715c44583d443d9ee24a31f2e0c753] to archive 2023-05-10 04:54:08,719 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1683694447035.e1dcbf091d2db6dbef8d701eb3d89914.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-05-10 04:54:08,720 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1683694447035.e1dcbf091d2db6dbef8d701eb3d89914.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e1dcbf091d2db6dbef8d701eb3d89914/cf2/8dc8ff2e94224e0bb3ca246e89c65dad.e0a753dfc7b1edf68920ae1543edc38c to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e1dcbf091d2db6dbef8d701eb3d89914/cf2/8dc8ff2e94224e0bb3ca246e89c65dad.e0a753dfc7b1edf68920ae1543edc38c 2023-05-10 04:54:08,721 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1683694447035.e1dcbf091d2db6dbef8d701eb3d89914.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e1dcbf091d2db6dbef8d701eb3d89914/cf2/bda778111963494983a9d36e39fdf23e.e0a753dfc7b1edf68920ae1543edc38c to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e1dcbf091d2db6dbef8d701eb3d89914/cf2/bda778111963494983a9d36e39fdf23e.e0a753dfc7b1edf68920ae1543edc38c 2023-05-10 04:54:08,723 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1683694447035.e1dcbf091d2db6dbef8d701eb3d89914.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e1dcbf091d2db6dbef8d701eb3d89914/cf2/c0c30c5caa4343a39ca186c265342dc5.e0a753dfc7b1edf68920ae1543edc38c to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e1dcbf091d2db6dbef8d701eb3d89914/cf2/c0c30c5caa4343a39ca186c265342dc5.e0a753dfc7b1edf68920ae1543edc38c 2023-05-10 04:54:08,723 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1683694447035.e1dcbf091d2db6dbef8d701eb3d89914.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e1dcbf091d2db6dbef8d701eb3d89914/cf2/8335cbbc49cf41f1982b7afcfeae3b15.e0a753dfc7b1edf68920ae1543edc38c to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e1dcbf091d2db6dbef8d701eb3d89914/cf2/8335cbbc49cf41f1982b7afcfeae3b15.e0a753dfc7b1edf68920ae1543edc38c 2023-05-10 04:54:08,724 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1683694447035.e1dcbf091d2db6dbef8d701eb3d89914.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e1dcbf091d2db6dbef8d701eb3d89914/cf2/testSplitTableRegionDeletedRowsDaughter=e0a753dfc7b1edf68920ae1543edc38c-f70a46cf983c4f61bd80c8814f0701eb to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e1dcbf091d2db6dbef8d701eb3d89914/cf2/testSplitTableRegionDeletedRowsDaughter=e0a753dfc7b1edf68920ae1543edc38c-f70a46cf983c4f61bd80c8814f0701eb 2023-05-10 04:54:08,725 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1683694447035.e1dcbf091d2db6dbef8d701eb3d89914.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e1dcbf091d2db6dbef8d701eb3d89914/cf2/testSplitTableRegionDeletedRowsDaughter=e0a753dfc7b1edf68920ae1543edc38c-8c9cb9ffa4ef41de964cc06282779ac0 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e1dcbf091d2db6dbef8d701eb3d89914/cf2/testSplitTableRegionDeletedRowsDaughter=e0a753dfc7b1edf68920ae1543edc38c-8c9cb9ffa4ef41de964cc06282779ac0 2023-05-10 04:54:08,726 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1683694447035.e1dcbf091d2db6dbef8d701eb3d89914.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e1dcbf091d2db6dbef8d701eb3d89914/cf2/testSplitTableRegionDeletedRowsDaughter=e0a753dfc7b1edf68920ae1543edc38c-57715c44583d443d9ee24a31f2e0c753 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e1dcbf091d2db6dbef8d701eb3d89914/cf2/testSplitTableRegionDeletedRowsDaughter=e0a753dfc7b1edf68920ae1543edc38c-57715c44583d443d9ee24a31f2e0c753 2023-05-10 04:54:08,729 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionDeletedRowsDaughter/e1dcbf091d2db6dbef8d701eb3d89914/recovered.edits/111.seqid, newMaxSeqId=111, maxSeqId=106 2023-05-10 04:54:08,730 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-10 04:54:08,730 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionDeletedRowsDaughter,60,1683694447035.e1dcbf091d2db6dbef8d701eb3d89914. 2023-05-10 04:54:08,730 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for e1dcbf091d2db6dbef8d701eb3d89914: 2023-05-10 04:54:08,731 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed e1dcbf091d2db6dbef8d701eb3d89914 2023-05-10 04:54:08,731 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=e1dcbf091d2db6dbef8d701eb3d89914, regionState=CLOSED 2023-05-10 04:54:08,732 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,60,1683694447035.e1dcbf091d2db6dbef8d701eb3d89914.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1683694448731"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683694448731"}]},"ts":"1683694448731"} 2023-05-10 04:54:08,734 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=82, resume processing ppid=81 2023-05-10 04:54:08,734 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=82, ppid=81, state=SUCCESS; CloseRegionProcedure e1dcbf091d2db6dbef8d701eb3d89914, server=jenkins-hbase4.apache.org,37995,1683694419499 in 203 msec 2023-05-10 04:54:08,736 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=81, resume processing ppid=79 2023-05-10 04:54:08,736 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=81, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=e1dcbf091d2db6dbef8d701eb3d89914, UNASSIGN in 206 msec 2023-05-10 04:54:08,737 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683694448737"}]},"ts":"1683694448737"} 2023-05-10 04:54:08,738 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=DISABLED in hbase:meta 2023-05-10 04:54:08,739 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionDeletedRowsDaughter to state=DISABLED 2023-05-10 04:54:08,741 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter in 219 msec 2023-05-10 04:54:08,826 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(1227): Checking to see if procedure is done pid=79 2023-05-10 04:54:08,827 INFO [Listener at localhost/45007] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionDeletedRowsDaughter, procId: 79 completed 2023-05-10 04:54:08,827 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:08,828 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] procedure2.ProcedureExecutor(1029): Stored pid=84, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:08,830 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=84, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:08,830 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=84, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:08,830 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(1227): Checking to see if procedure is done pid=84 2023-05-10 04:54:08,835 DEBUG [HFileArchiver-8] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c 2023-05-10 04:54:08,835 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/08557250442eebc7c86235cdf682b1a9 2023-05-10 04:54:08,835 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e1dcbf091d2db6dbef8d701eb3d89914 2023-05-10 04:54:08,838 DEBUG [HFileArchiver-8] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf1, FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf2, FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/recovered.edits] 2023-05-10 04:54:08,838 DEBUG [HFileArchiver-2] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/08557250442eebc7c86235cdf682b1a9/cf1, FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/08557250442eebc7c86235cdf682b1a9/cf2, FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/08557250442eebc7c86235cdf682b1a9/recovered.edits] 2023-05-10 04:54:08,838 DEBUG [HFileArchiver-4] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e1dcbf091d2db6dbef8d701eb3d89914/cf1, FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e1dcbf091d2db6dbef8d701eb3d89914/cf2, FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e1dcbf091d2db6dbef8d701eb3d89914/recovered.edits] 2023-05-10 04:54:08,842 DEBUG [HFileArchiver-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/08557250442eebc7c86235cdf682b1a9/cf1/bc79127118d74a76a5be90840e50d848 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/08557250442eebc7c86235cdf682b1a9/cf1/bc79127118d74a76a5be90840e50d848 2023-05-10 04:54:08,842 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e1dcbf091d2db6dbef8d701eb3d89914/cf1/21c6268577fd48f6b60d6f4a64b0df16 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e1dcbf091d2db6dbef8d701eb3d89914/cf1/21c6268577fd48f6b60d6f4a64b0df16 2023-05-10 04:54:08,842 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf1/069e6767796b4374972ea4f62b4268aa to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf1/069e6767796b4374972ea4f62b4268aa 2023-05-10 04:54:08,843 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf1/4391ef9079694b3d9c4aa820517a211f to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf1/4391ef9079694b3d9c4aa820517a211f 2023-05-10 04:54:08,845 DEBUG [HFileArchiver-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/08557250442eebc7c86235cdf682b1a9/cf2/de25529ed5d44f04be9f70f3fc3deb08 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/08557250442eebc7c86235cdf682b1a9/cf2/de25529ed5d44f04be9f70f3fc3deb08 2023-05-10 04:54:08,845 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf1/4916dab2636c42c190c9d9d879485e00 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf1/4916dab2636c42c190c9d9d879485e00 2023-05-10 04:54:08,845 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e1dcbf091d2db6dbef8d701eb3d89914/cf2/49afb61cf8544b59ac7da765b1018b1c to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e1dcbf091d2db6dbef8d701eb3d89914/cf2/49afb61cf8544b59ac7da765b1018b1c 2023-05-10 04:54:08,847 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf1/5c92186c363e4e2ea144b06cf2ef4148 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf1/5c92186c363e4e2ea144b06cf2ef4148 2023-05-10 04:54:08,848 DEBUG [HFileArchiver-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/08557250442eebc7c86235cdf682b1a9/recovered.edits/113.seqid to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/08557250442eebc7c86235cdf682b1a9/recovered.edits/113.seqid 2023-05-10 04:54:08,848 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf1/6abf15e207cc4ef7bb40e895775d046c to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf1/6abf15e207cc4ef7bb40e895775d046c 2023-05-10 04:54:08,849 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e1dcbf091d2db6dbef8d701eb3d89914/recovered.edits/111.seqid to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e1dcbf091d2db6dbef8d701eb3d89914/recovered.edits/111.seqid 2023-05-10 04:54:08,849 DEBUG [HFileArchiver-2] backup.HFileArchiver(596): Deleted hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/08557250442eebc7c86235cdf682b1a9 2023-05-10 04:54:08,849 DEBUG [HFileArchiver-4] backup.HFileArchiver(596): Deleted hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e1dcbf091d2db6dbef8d701eb3d89914 2023-05-10 04:54:08,850 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf1/6fcee43348954f60b12fefd67f843fe5 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf1/6fcee43348954f60b12fefd67f843fe5 2023-05-10 04:54:08,851 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf1/9c5dc820063c4ac1b94e51a619cf0ca3 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf1/9c5dc820063c4ac1b94e51a619cf0ca3 2023-05-10 04:54:08,852 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf1/aab44d0818ef4a9791980635a3fffd5d to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf1/aab44d0818ef4a9791980635a3fffd5d 2023-05-10 04:54:08,853 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf1/b9ae488cd59742d9bf602451afc29300 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf1/b9ae488cd59742d9bf602451afc29300 2023-05-10 04:54:08,854 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf1/bee3c0539163404a9c5862a802d06fc8 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf1/bee3c0539163404a9c5862a802d06fc8 2023-05-10 04:54:08,856 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf2/57715c44583d443d9ee24a31f2e0c753 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf2/57715c44583d443d9ee24a31f2e0c753 2023-05-10 04:54:08,858 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf2/8335cbbc49cf41f1982b7afcfeae3b15 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf2/8335cbbc49cf41f1982b7afcfeae3b15 2023-05-10 04:54:08,859 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf2/839ff9ffc87b4074b7e77f7b6e45b9ce to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf2/839ff9ffc87b4074b7e77f7b6e45b9ce 2023-05-10 04:54:08,860 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf2/8c9cb9ffa4ef41de964cc06282779ac0 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf2/8c9cb9ffa4ef41de964cc06282779ac0 2023-05-10 04:54:08,862 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf2/8dc8ff2e94224e0bb3ca246e89c65dad to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf2/8dc8ff2e94224e0bb3ca246e89c65dad 2023-05-10 04:54:08,863 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf2/bda778111963494983a9d36e39fdf23e to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf2/bda778111963494983a9d36e39fdf23e 2023-05-10 04:54:08,864 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf2/c0c30c5caa4343a39ca186c265342dc5 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf2/c0c30c5caa4343a39ca186c265342dc5 2023-05-10 04:54:08,865 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf2/e8a131ba8e924d2ab9e7dd5b50323810 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf2/e8a131ba8e924d2ab9e7dd5b50323810 2023-05-10 04:54:08,866 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf2/f70a46cf983c4f61bd80c8814f0701eb to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf2/f70a46cf983c4f61bd80c8814f0701eb 2023-05-10 04:54:08,868 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf2/fcb908e83d9d411482f19b7258cd7021 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/cf2/fcb908e83d9d411482f19b7258cd7021 2023-05-10 04:54:08,870 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/recovered.edits/106.seqid to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c/recovered.edits/106.seqid 2023-05-10 04:54:08,870 DEBUG [HFileArchiver-8] backup.HFileArchiver(596): Deleted hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/e0a753dfc7b1edf68920ae1543edc38c 2023-05-10 04:54:08,870 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionDeletedRowsDaughter regions 2023-05-10 04:54:08,872 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=84, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:08,874 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionDeletedRowsDaughter from hbase:meta 2023-05-10 04:54:08,876 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionDeletedRowsDaughter' descriptor. 2023-05-10 04:54:08,877 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=84, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:08,877 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionDeletedRowsDaughter' from region states. 2023-05-10 04:54:08,877 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter,,1683694444835.e0a753dfc7b1edf68920ae1543edc38c.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1683694448877"}]},"ts":"9223372036854775807"} 2023-05-10 04:54:08,877 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter,,1683694447035.08557250442eebc7c86235cdf682b1a9.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1683694448877"}]},"ts":"9223372036854775807"} 2023-05-10 04:54:08,877 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter,60,1683694447035.e1dcbf091d2db6dbef8d701eb3d89914.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1683694448877"}]},"ts":"9223372036854775807"} 2023-05-10 04:54:08,878 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-05-10 04:54:08,878 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => e0a753dfc7b1edf68920ae1543edc38c, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1683694444835.e0a753dfc7b1edf68920ae1543edc38c.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 08557250442eebc7c86235cdf682b1a9, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1683694447035.08557250442eebc7c86235cdf682b1a9.', STARTKEY => '', ENDKEY => '60'}, {ENCODED => e1dcbf091d2db6dbef8d701eb3d89914, NAME => 'testSplitTableRegionDeletedRowsDaughter,60,1683694447035.e1dcbf091d2db6dbef8d701eb3d89914.', STARTKEY => '60', ENDKEY => ''}] 2023-05-10 04:54:08,878 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionDeletedRowsDaughter' as deleted. 2023-05-10 04:54:08,879 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1683694448878"}]},"ts":"9223372036854775807"} 2023-05-10 04:54:08,880 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionDeletedRowsDaughter state from META 2023-05-10 04:54:08,885 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=84, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:54:08,886 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter in 58 msec 2023-05-10 04:54:08,931 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(1227): Checking to see if procedure is done pid=84 2023-05-10 04:54:08,932 INFO [Listener at localhost/45007] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionDeletedRowsDaughter, procId: 84 completed 2023-05-10 04:54:08,946 INFO [Listener at localhost/45007] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionDeletedRowsDaughter Thread=412 (was 403) Potentially hanging thread: hconnection-0x248fdd40-shared-pool-18 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x248fdd40-shared-pool-15 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x248fdd40-metaLookup-shared--pool-11 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-2011914893_15 at /127.0.0.1:54322 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-2011914893_15 at /127.0.0.1:54304 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x248fdd40-shared-pool-13 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-2011914893_15 at /127.0.0.1:51308 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-2011914893_15 at /127.0.0.1:51294 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-5 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x248fdd40-metaLookup-shared--pool-12 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x248fdd40-shared-pool-17 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-2011914893_15 at /127.0.0.1:51378 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-2011914893_15 at /127.0.0.1:51314 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-2011914893_15 at /127.0.0.1:51340 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-2011914893_15 at /127.0.0.1:51282 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x248fdd40-shared-pool-16 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-2011914893_15 at /127.0.0.1:51292 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x248fdd40-metaLookup-shared--pool-13 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-2011914893_15 at /127.0.0.1:54316 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-2011914893_15 at /127.0.0.1:54272 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x248fdd40-shared-pool-14 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-2011914893_15 at /127.0.0.1:51264 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-2011914893_15 at /127.0.0.1:51286 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-2011914893_15 at /127.0.0.1:54328 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:1;jenkins-hbase4:37995-shortCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-2011914893_15 at /127.0.0.1:54262 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x248fdd40-metaLookup-shared--pool-9 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-2011914893_15 at /127.0.0.1:54294 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x248fdd40-metaLookup-shared--pool-10 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=868 (was 849) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=323 (was 308) - SystemLoadAverage LEAK? -, ProcessCount=175 (was 175), AvailableMemoryMB=9627 (was 9648) 2023-05-10 04:54:08,960 INFO [Listener at localhost/45007] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testRollbackForSplitTableRegionWithReplica Thread=412, OpenFileDescriptor=868, MaxFileDescriptor=60000, SystemLoadAverage=323, ProcessCount=175, AvailableMemoryMB=9626 2023-05-10 04:54:08,960 WARN [Listener at localhost/45007] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-05-10 04:54:08,967 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-05-10 04:54:08,968 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-05-10 04:54:08,969 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(1029): Stored pid=85, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-10 04:54:08,970 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_PRE_OPERATION 2023-05-10 04:54:08,971 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-05-10 04:54:08,972 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/d2b34babccb97d3c1ac066d9e90ae654 2023-05-10 04:54:08,973 DEBUG [HFileArchiver-5] backup.HFileArchiver(153): Directory hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/d2b34babccb97d3c1ac066d9e90ae654 empty. 2023-05-10 04:54:08,973 DEBUG [HFileArchiver-5] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/d2b34babccb97d3c1ac066d9e90ae654 2023-05-10 04:54:08,973 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackForSplitTableRegionWithReplica regions 2023-05-10 04:54:08,983 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000001 2023-05-10 04:54:08,984 INFO [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(7675): creating {ENCODED => d2b34babccb97d3c1ac066d9e90ae654, NAME => 'testRollbackForSplitTableRegionWithReplica,,1683694448968.d2b34babccb97d3c1ac066d9e90ae654.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testRollbackForSplitTableRegionWithReplica', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp 2023-05-10 04:54:08,993 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(866): Instantiated testRollbackForSplitTableRegionWithReplica,,1683694448968.d2b34babccb97d3c1ac066d9e90ae654.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-10 04:54:08,993 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1604): Closing d2b34babccb97d3c1ac066d9e90ae654, disabling compactions & flushes 2023-05-10 04:54:08,993 INFO [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1626): Closing region testRollbackForSplitTableRegionWithReplica,,1683694448968.d2b34babccb97d3c1ac066d9e90ae654. 2023-05-10 04:54:08,993 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1683694448968.d2b34babccb97d3c1ac066d9e90ae654. 2023-05-10 04:54:08,993 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1714): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1683694448968.d2b34babccb97d3c1ac066d9e90ae654. after waiting 0 ms 2023-05-10 04:54:08,993 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1724): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1683694448968.d2b34babccb97d3c1ac066d9e90ae654. 2023-05-10 04:54:08,993 INFO [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1838): Closed testRollbackForSplitTableRegionWithReplica,,1683694448968.d2b34babccb97d3c1ac066d9e90ae654. 2023-05-10 04:54:08,993 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1558): Region close journal for d2b34babccb97d3c1ac066d9e90ae654: 2023-05-10 04:54:08,995 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_ADD_TO_META 2023-05-10 04:54:08,996 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1683694448968.d2b34babccb97d3c1ac066d9e90ae654.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1683694448996"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683694448996"}]},"ts":"1683694448996"} 2023-05-10 04:54:08,997 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-05-10 04:54:08,997 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-05-10 04:54:08,998 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683694448997"}]},"ts":"1683694448997"} 2023-05-10 04:54:08,998 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=ENABLING in hbase:meta 2023-05-10 04:54:09,001 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-05-10 04:54:09,001 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-05-10 04:54:09,001 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-05-10 04:54:09,002 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-05-10 04:54:09,002 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-05-10 04:54:09,002 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=86, ppid=85, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=d2b34babccb97d3c1ac066d9e90ae654, ASSIGN}] 2023-05-10 04:54:09,003 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=86, ppid=85, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=d2b34babccb97d3c1ac066d9e90ae654, ASSIGN 2023-05-10 04:54:09,003 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=86, ppid=85, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=d2b34babccb97d3c1ac066d9e90ae654, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,37823,1683694419534; forceNewPlan=false, retain=false 2023-05-10 04:54:09,154 INFO [jenkins-hbase4:37903] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-05-10 04:54:09,156 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=86 updating hbase:meta row=d2b34babccb97d3c1ac066d9e90ae654, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:54:09,156 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1683694448968.d2b34babccb97d3c1ac066d9e90ae654.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1683694449156"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683694449156"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683694449156"}]},"ts":"1683694449156"} 2023-05-10 04:54:09,158 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=87, ppid=86, state=RUNNABLE; OpenRegionProcedure d2b34babccb97d3c1ac066d9e90ae654, server=jenkins-hbase4.apache.org,37823,1683694419534}] 2023-05-10 04:54:09,312 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testRollbackForSplitTableRegionWithReplica,,1683694448968.d2b34babccb97d3c1ac066d9e90ae654. 2023-05-10 04:54:09,312 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => d2b34babccb97d3c1ac066d9e90ae654, NAME => 'testRollbackForSplitTableRegionWithReplica,,1683694448968.d2b34babccb97d3c1ac066d9e90ae654.', STARTKEY => '', ENDKEY => ''} 2023-05-10 04:54:09,312 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-05-10 04:54:09,313 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackForSplitTableRegionWithReplica d2b34babccb97d3c1ac066d9e90ae654 2023-05-10 04:54:09,313 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testRollbackForSplitTableRegionWithReplica,,1683694448968.d2b34babccb97d3c1ac066d9e90ae654.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-10 04:54:09,313 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for d2b34babccb97d3c1ac066d9e90ae654 2023-05-10 04:54:09,313 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for d2b34babccb97d3c1ac066d9e90ae654 2023-05-10 04:54:09,314 INFO [StoreOpener-d2b34babccb97d3c1ac066d9e90ae654-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region d2b34babccb97d3c1ac066d9e90ae654 2023-05-10 04:54:09,315 DEBUG [StoreOpener-d2b34babccb97d3c1ac066d9e90ae654-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackForSplitTableRegionWithReplica/d2b34babccb97d3c1ac066d9e90ae654/cf1 2023-05-10 04:54:09,315 DEBUG [StoreOpener-d2b34babccb97d3c1ac066d9e90ae654-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackForSplitTableRegionWithReplica/d2b34babccb97d3c1ac066d9e90ae654/cf1 2023-05-10 04:54:09,315 INFO [StoreOpener-d2b34babccb97d3c1ac066d9e90ae654-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d2b34babccb97d3c1ac066d9e90ae654 columnFamilyName cf1 2023-05-10 04:54:09,316 INFO [StoreOpener-d2b34babccb97d3c1ac066d9e90ae654-1] regionserver.HStore(310): Store=d2b34babccb97d3c1ac066d9e90ae654/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-10 04:54:09,317 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackForSplitTableRegionWithReplica/d2b34babccb97d3c1ac066d9e90ae654 2023-05-10 04:54:09,317 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackForSplitTableRegionWithReplica/d2b34babccb97d3c1ac066d9e90ae654 2023-05-10 04:54:09,319 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for d2b34babccb97d3c1ac066d9e90ae654 2023-05-10 04:54:09,321 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackForSplitTableRegionWithReplica/d2b34babccb97d3c1ac066d9e90ae654/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-05-10 04:54:09,321 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened d2b34babccb97d3c1ac066d9e90ae654; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59028995, jitterRate=-0.12039943039417267}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-05-10 04:54:09,321 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for d2b34babccb97d3c1ac066d9e90ae654: 2023-05-10 04:54:09,322 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2334): Post open deploy tasks for testRollbackForSplitTableRegionWithReplica,,1683694448968.d2b34babccb97d3c1ac066d9e90ae654., pid=87, masterSystemTime=1683694449309 2023-05-10 04:54:09,323 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2361): Finished post open deploy task for testRollbackForSplitTableRegionWithReplica,,1683694448968.d2b34babccb97d3c1ac066d9e90ae654. 2023-05-10 04:54:09,324 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testRollbackForSplitTableRegionWithReplica,,1683694448968.d2b34babccb97d3c1ac066d9e90ae654. 2023-05-10 04:54:09,324 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=86 updating hbase:meta row=d2b34babccb97d3c1ac066d9e90ae654, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:54:09,324 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackForSplitTableRegionWithReplica,,1683694448968.d2b34babccb97d3c1ac066d9e90ae654.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1683694449324"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1683694449324"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1683694449324"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683694449324"}]},"ts":"1683694449324"} 2023-05-10 04:54:09,326 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=87, resume processing ppid=86 2023-05-10 04:54:09,326 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=87, ppid=86, state=SUCCESS; OpenRegionProcedure d2b34babccb97d3c1ac066d9e90ae654, server=jenkins-hbase4.apache.org,37823,1683694419534 in 167 msec 2023-05-10 04:54:09,328 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=86, resume processing ppid=85 2023-05-10 04:54:09,328 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=86, ppid=85, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=d2b34babccb97d3c1ac066d9e90ae654, ASSIGN in 325 msec 2023-05-10 04:54:09,329 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-05-10 04:54:09,329 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683694449329"}]},"ts":"1683694449329"} 2023-05-10 04:54:09,330 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=ENABLED in hbase:meta 2023-05-10 04:54:09,339 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_POST_OPERATION 2023-05-10 04:54:09,340 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica in 371 msec 2023-05-10 04:54:09,478 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.HMaster$12(2454): Client=jenkins//172.31.14.131 modify table testRollbackForSplitTableRegionWithReplica from 'testRollbackForSplitTableRegionWithReplica', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} to 'testRollbackForSplitTableRegionWithReplica', {TABLE_ATTRIBUTES => {REGION_REPLICATION => '2'}}, {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-05-10 04:54:09,485 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] procedure2.ProcedureExecutor(1029): Stored pid=88, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-10 04:54:09,498 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000002 2023-05-10 04:54:09,499 DEBUG [PEWorker-1] util.FSTableDescriptors(519): Deleted hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000001 2023-05-10 04:54:09,499 INFO [PEWorker-1] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000002 2023-05-10 04:54:09,503 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=89, ppid=88, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testRollbackForSplitTableRegionWithReplica}] 2023-05-10 04:54:09,514 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=90, ppid=89, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=d2b34babccb97d3c1ac066d9e90ae654, REOPEN/MOVE}] 2023-05-10 04:54:09,515 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=90, ppid=89, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=d2b34babccb97d3c1ac066d9e90ae654, REOPEN/MOVE 2023-05-10 04:54:09,516 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=d2b34babccb97d3c1ac066d9e90ae654, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:54:09,516 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1683694448968.d2b34babccb97d3c1ac066d9e90ae654.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1683694449516"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683694449516"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683694449516"}]},"ts":"1683694449516"} 2023-05-10 04:54:09,517 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=91, ppid=90, state=RUNNABLE; CloseRegionProcedure d2b34babccb97d3c1ac066d9e90ae654, server=jenkins-hbase4.apache.org,37823,1683694419534}] 2023-05-10 04:54:09,669 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close d2b34babccb97d3c1ac066d9e90ae654 2023-05-10 04:54:09,670 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing d2b34babccb97d3c1ac066d9e90ae654, disabling compactions & flushes 2023-05-10 04:54:09,670 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testRollbackForSplitTableRegionWithReplica,,1683694448968.d2b34babccb97d3c1ac066d9e90ae654. 2023-05-10 04:54:09,670 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1683694448968.d2b34babccb97d3c1ac066d9e90ae654. 2023-05-10 04:54:09,670 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1683694448968.d2b34babccb97d3c1ac066d9e90ae654. after waiting 0 ms 2023-05-10 04:54:09,670 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1683694448968.d2b34babccb97d3c1ac066d9e90ae654. 2023-05-10 04:54:09,673 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackForSplitTableRegionWithReplica/d2b34babccb97d3c1ac066d9e90ae654/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-05-10 04:54:09,674 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-10 04:54:09,674 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testRollbackForSplitTableRegionWithReplica,,1683694448968.d2b34babccb97d3c1ac066d9e90ae654. 2023-05-10 04:54:09,674 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for d2b34babccb97d3c1ac066d9e90ae654: 2023-05-10 04:54:09,674 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(3508): Not adding moved region record: d2b34babccb97d3c1ac066d9e90ae654 to self. 2023-05-10 04:54:09,675 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed d2b34babccb97d3c1ac066d9e90ae654 2023-05-10 04:54:09,676 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=d2b34babccb97d3c1ac066d9e90ae654, regionState=CLOSED 2023-05-10 04:54:09,676 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1683694448968.d2b34babccb97d3c1ac066d9e90ae654.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1683694449676"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683694449676"}]},"ts":"1683694449676"} 2023-05-10 04:54:09,678 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=91, resume processing ppid=90 2023-05-10 04:54:09,679 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=91, ppid=90, state=SUCCESS; CloseRegionProcedure d2b34babccb97d3c1ac066d9e90ae654, server=jenkins-hbase4.apache.org,37823,1683694419534 in 160 msec 2023-05-10 04:54:09,679 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=90, ppid=89, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=d2b34babccb97d3c1ac066d9e90ae654, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase4.apache.org,37823,1683694419534; forceNewPlan=false, retain=true 2023-05-10 04:54:09,829 INFO [jenkins-hbase4:37903] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-05-10 04:54:09,830 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=d2b34babccb97d3c1ac066d9e90ae654, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:54:09,830 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1683694448968.d2b34babccb97d3c1ac066d9e90ae654.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1683694449830"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683694449830"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683694449830"}]},"ts":"1683694449830"} 2023-05-10 04:54:09,831 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=92, ppid=90, state=RUNNABLE; OpenRegionProcedure d2b34babccb97d3c1ac066d9e90ae654, server=jenkins-hbase4.apache.org,37823,1683694419534}] 2023-05-10 04:54:09,986 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testRollbackForSplitTableRegionWithReplica,,1683694448968.d2b34babccb97d3c1ac066d9e90ae654. 2023-05-10 04:54:09,986 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => d2b34babccb97d3c1ac066d9e90ae654, NAME => 'testRollbackForSplitTableRegionWithReplica,,1683694448968.d2b34babccb97d3c1ac066d9e90ae654.', STARTKEY => '', ENDKEY => ''} 2023-05-10 04:54:09,986 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-05-10 04:54:09,986 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackForSplitTableRegionWithReplica d2b34babccb97d3c1ac066d9e90ae654 2023-05-10 04:54:09,986 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testRollbackForSplitTableRegionWithReplica,,1683694448968.d2b34babccb97d3c1ac066d9e90ae654.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-10 04:54:09,986 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for d2b34babccb97d3c1ac066d9e90ae654 2023-05-10 04:54:09,986 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for d2b34babccb97d3c1ac066d9e90ae654 2023-05-10 04:54:09,987 INFO [StoreOpener-d2b34babccb97d3c1ac066d9e90ae654-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region d2b34babccb97d3c1ac066d9e90ae654 2023-05-10 04:54:09,988 DEBUG [StoreOpener-d2b34babccb97d3c1ac066d9e90ae654-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackForSplitTableRegionWithReplica/d2b34babccb97d3c1ac066d9e90ae654/cf1 2023-05-10 04:54:09,988 DEBUG [StoreOpener-d2b34babccb97d3c1ac066d9e90ae654-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackForSplitTableRegionWithReplica/d2b34babccb97d3c1ac066d9e90ae654/cf1 2023-05-10 04:54:09,989 INFO [StoreOpener-d2b34babccb97d3c1ac066d9e90ae654-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d2b34babccb97d3c1ac066d9e90ae654 columnFamilyName cf1 2023-05-10 04:54:09,989 INFO [StoreOpener-d2b34babccb97d3c1ac066d9e90ae654-1] regionserver.HStore(310): Store=d2b34babccb97d3c1ac066d9e90ae654/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-10 04:54:09,990 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackForSplitTableRegionWithReplica/d2b34babccb97d3c1ac066d9e90ae654 2023-05-10 04:54:09,991 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackForSplitTableRegionWithReplica/d2b34babccb97d3c1ac066d9e90ae654 2023-05-10 04:54:09,993 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for d2b34babccb97d3c1ac066d9e90ae654 2023-05-10 04:54:09,993 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened d2b34babccb97d3c1ac066d9e90ae654; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=73541957, jitterRate=0.095860555768013}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-05-10 04:54:09,993 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for d2b34babccb97d3c1ac066d9e90ae654: 2023-05-10 04:54:09,994 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2334): Post open deploy tasks for testRollbackForSplitTableRegionWithReplica,,1683694448968.d2b34babccb97d3c1ac066d9e90ae654., pid=92, masterSystemTime=1683694449983 2023-05-10 04:54:09,995 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2361): Finished post open deploy task for testRollbackForSplitTableRegionWithReplica,,1683694448968.d2b34babccb97d3c1ac066d9e90ae654. 2023-05-10 04:54:09,995 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testRollbackForSplitTableRegionWithReplica,,1683694448968.d2b34babccb97d3c1ac066d9e90ae654. 2023-05-10 04:54:09,996 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=d2b34babccb97d3c1ac066d9e90ae654, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:54:09,996 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackForSplitTableRegionWithReplica,,1683694448968.d2b34babccb97d3c1ac066d9e90ae654.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1683694449996"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1683694449996"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1683694449996"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683694449996"}]},"ts":"1683694449996"} 2023-05-10 04:54:09,998 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=92, resume processing ppid=90 2023-05-10 04:54:09,998 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=92, ppid=90, state=SUCCESS; OpenRegionProcedure d2b34babccb97d3c1ac066d9e90ae654, server=jenkins-hbase4.apache.org,37823,1683694419534 in 166 msec 2023-05-10 04:54:09,999 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=90, resume processing ppid=89 2023-05-10 04:54:10,000 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=90, ppid=89, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=d2b34babccb97d3c1ac066d9e90ae654, REOPEN/MOVE in 484 msec 2023-05-10 04:54:10,002 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=89, resume processing ppid=88 2023-05-10 04:54:10,002 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=89, ppid=88, state=SUCCESS; ReopenTableRegionsProcedure table=testRollbackForSplitTableRegionWithReplica in 499 msec 2023-05-10 04:54:10,005 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=93, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=bddd3bfc356670bc9cd9492c47d95a01, ASSIGN}] 2023-05-10 04:54:10,006 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=93, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=bddd3bfc356670bc9cd9492c47d95a01, ASSIGN 2023-05-10 04:54:10,006 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=93, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=bddd3bfc356670bc9cd9492c47d95a01, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-05-10 04:54:10,157 DEBUG [jenkins-hbase4:37903] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-05-10 04:54:10,158 DEBUG [jenkins-hbase4:37903] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-05-10 04:54:10,158 DEBUG [jenkins-hbase4:37903] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-05-10 04:54:10,158 DEBUG [jenkins-hbase4:37903] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-05-10 04:54:10,158 DEBUG [jenkins-hbase4:37903] balancer.BaseLoadBalancer$Cluster(378): Number of tables=3, number of hosts=1, number of racks=1 2023-05-10 04:54:10,159 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=93 updating hbase:meta row=bddd3bfc356670bc9cd9492c47d95a01, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:54:10,159 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1683694448968.d2b34babccb97d3c1ac066d9e90ae654.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1683694450159"},{"qualifier":"sn_0001","vlen":45,"tag":[],"timestamp":"1683694450159"},{"qualifier":"state_0001","vlen":7,"tag":[],"timestamp":"1683694450159"}]},"ts":"1683694450159"} 2023-05-10 04:54:10,163 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=94, ppid=93, state=RUNNABLE; OpenRegionProcedure bddd3bfc356670bc9cd9492c47d95a01, server=jenkins-hbase4.apache.org,37995,1683694419499}] 2023-05-10 04:54:10,317 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testRollbackForSplitTableRegionWithReplica,,1683694448968_0001.bddd3bfc356670bc9cd9492c47d95a01. 2023-05-10 04:54:10,317 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => bddd3bfc356670bc9cd9492c47d95a01, NAME => 'testRollbackForSplitTableRegionWithReplica,,1683694448968_0001.bddd3bfc356670bc9cd9492c47d95a01.', STARTKEY => '', ENDKEY => '', REPLICA_ID => 1} 2023-05-10 04:54:10,317 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-05-10 04:54:10,317 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackForSplitTableRegionWithReplica bddd3bfc356670bc9cd9492c47d95a01 2023-05-10 04:54:10,317 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testRollbackForSplitTableRegionWithReplica,,1683694448968_0001.bddd3bfc356670bc9cd9492c47d95a01.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-10 04:54:10,318 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for bddd3bfc356670bc9cd9492c47d95a01 2023-05-10 04:54:10,318 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for bddd3bfc356670bc9cd9492c47d95a01 2023-05-10 04:54:10,318 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:10,418 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:10,518 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:10,618 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:10,719 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:10,819 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:10,919 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:10,972 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testRollbackForSplitTableRegionWithReplica' 2023-05-10 04:54:11,019 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:11,119 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:11,219 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:11,320 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:11,420 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:11,520 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:11,620 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:11,720 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:11,821 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:11,921 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:12,021 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:12,121 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:12,221 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:12,322 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:12,422 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:12,522 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:12,622 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:12,723 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:12,736 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-05-10 04:54:12,823 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:12,923 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:13,023 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:13,123 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:13,224 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:13,324 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:13,424 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:13,524 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:13,624 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:13,724 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:13,825 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:13,925 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:14,025 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:14,125 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:14,226 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:14,326 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:14,426 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:14,526 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:14,626 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:14,727 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:14,827 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:14,927 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:15,027 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:15,127 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:15,228 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:15,328 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:15,428 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:15,528 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:15,628 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:15,729 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:15,829 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:15,929 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:16,029 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:16,129 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:16,230 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:16,330 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:16,430 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:16,530 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:16,630 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:16,731 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:16,831 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:16,931 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:17,031 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:17,131 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:17,232 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:17,332 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:17,432 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:17,532 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:17,632 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:17,733 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:17,833 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:17,933 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:18,033 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:18,133 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:18,234 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:18,334 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:18,434 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:18,534 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:18,634 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:18,735 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:18,835 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:18,935 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:19,035 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:19,136 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:19,236 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:19,336 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:19,436 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:19,536 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:19,637 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:19,737 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:19,837 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:19,937 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:20,037 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:20,138 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:20,238 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:20,338 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:20,438 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:20,539 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:20,639 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:20,739 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:20,839 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:20,940 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:21,040 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:21,140 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:21,240 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:21,340 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:21,441 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:21,541 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:21,641 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:21,741 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:21,841 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:21,942 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:22,042 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:22,142 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:22,242 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:22,342 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:22,443 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:22,543 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:22,643 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:22,743 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:22,843 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:22,943 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:23,044 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:23,144 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:23,244 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:23,344 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:23,444 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:23,545 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:23,645 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:23,745 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:23,845 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:23,946 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:24,046 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:24,146 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:24,246 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:24,346 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:24,447 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:24,547 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:24,647 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:24,747 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:24,847 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:24,948 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:25,048 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:25,148 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:25,248 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:25,348 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:25,449 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:25,549 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:25,649 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:25,749 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:25,850 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:25,950 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:26,050 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:26,150 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:26,250 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:26,351 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:26,451 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:26,551 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:26,651 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:26,751 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:26,852 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:26,952 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:27,052 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:27,152 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:27,253 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:27,353 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:27,453 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:27,553 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:27,654 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:27,754 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:27,854 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:27,954 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:28,054 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:28,155 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:28,255 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:28,355 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:28,455 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:28,555 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:28,656 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:28,756 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:28,856 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:28,956 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:29,056 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:29,157 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:29,257 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:29,357 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:29,457 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:29,557 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:29,658 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:29,758 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:29,858 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:29,959 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:30,059 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:30,159 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:30,259 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:30,359 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:30,460 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:30,560 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:30,660 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:30,760 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:30,860 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:30,961 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:31,061 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:31,161 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:31,261 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:31,361 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:31,462 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:31,562 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:31,662 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:31,762 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:31,863 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:31,963 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:32,063 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:32,163 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:32,263 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:32,364 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:32,464 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:32,564 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:32,664 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:32,765 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:32,865 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:32,965 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:33,066 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:33,166 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:33,266 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:33,366 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:33,466 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:33,567 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:33,667 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:33,767 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:33,867 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:33,968 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:34,068 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:34,168 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:34,268 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:34,368 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:34,469 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:34,569 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:34,669 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:34,769 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:34,870 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:34,970 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:35,070 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:35,170 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:35,270 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:35,371 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:35,471 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:35,571 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:35,671 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:35,772 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:35,872 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:35,972 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:36,072 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:36,173 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:36,273 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:36,373 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:36,473 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:36,573 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:36,674 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:36,774 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:36,874 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:36,974 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:37,074 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:37,175 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:37,275 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:37,375 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:37,475 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:37,575 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:37,675 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:37,776 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:37,876 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:37,976 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:38,076 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:38,177 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:38,277 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:38,377 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:38,478 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:38,578 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:38,678 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:38,778 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:38,878 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:38,979 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:39,079 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:39,179 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:39,279 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:39,380 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:39,480 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:39,580 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:39,680 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:39,780 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:39,880 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:39,981 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:40,081 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:40,181 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:40,281 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:40,382 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:40,482 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:40,582 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:40,682 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:40,782 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:40,883 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:40,983 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:41,083 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:41,183 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:41,283 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:41,384 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:41,484 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:41,584 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:41,684 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:41,785 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:41,885 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:41,985 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:42,085 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:42,186 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:42,286 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:42,386 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:42,486 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:42,586 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:42,687 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:42,787 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:42,887 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:42,987 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:43,087 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:43,188 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:43,288 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:43,388 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:43,488 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:43,588 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:43,688 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:43,789 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:43,889 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:43,989 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:44,089 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:44,190 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:44,290 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:44,390 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:44,490 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:44,590 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:44,691 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:44,791 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:44,891 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:44,991 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:45,091 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:45,191 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:45,292 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:45,392 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:45,492 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:45,592 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:45,692 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:45,793 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:45,893 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:45,993 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:46,093 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:46,194 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:46,294 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:46,394 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:46,494 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:46,594 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:46,695 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:46,795 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:46,895 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:46,995 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:47,095 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:47,195 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:47,296 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:47,396 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:47,496 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:47,596 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:47,696 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:47,797 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:47,897 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:47,997 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:48,097 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:48,198 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:48,298 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:48,398 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:48,498 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:48,598 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:48,699 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:48,799 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:48,899 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:48,999 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:49,099 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:49,200 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:49,300 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:49,400 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:49,500 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:49,600 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:49,700 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:49,801 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:49,901 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:50,001 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:50,101 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:50,201 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:50,302 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:50,402 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:50,502 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:50,602 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:50,702 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:50,803 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:50,903 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:51,003 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:51,103 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:51,203 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:51,304 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:51,404 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:51,504 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:51,604 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:51,704 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:51,805 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:51,905 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:52,005 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:52,105 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:52,206 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:52,306 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:52,406 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:52,506 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:52,606 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:52,707 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:52,807 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:52,907 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:53,007 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:53,107 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:53,208 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:53,308 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:53,408 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:53,508 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:53,608 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:53,708 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:53,809 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:53,909 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:54,009 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:54,109 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:54,209 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:54,310 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:54,410 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:54,510 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:54,610 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:54,710 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:54,811 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:54,911 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:55,011 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:55,111 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:55,211 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:55,312 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:55,412 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:55,512 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:55,612 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:55,712 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:55,813 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:55,913 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:56,013 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:56,113 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:56,213 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:56,314 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:56,414 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:56,514 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:56,614 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:56,714 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:56,815 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:56,915 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:57,015 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:57,115 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:57,215 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:57,316 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:57,416 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:57,516 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:57,616 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:57,716 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:57,817 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:57,917 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:58,017 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:58,117 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:58,217 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:58,317 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:58,418 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:58,518 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:58,618 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:58,718 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:58,818 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:58,919 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:59,019 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:59,119 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:59,219 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:59,319 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:59,420 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:59,520 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:59,620 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:59,720 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:59,820 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:54:59,921 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:00,021 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:00,121 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:00,221 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:00,321 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:00,422 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:00,522 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:00,622 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:00,722 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:00,822 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:00,923 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:01,023 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:01,123 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:01,223 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:01,323 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:01,424 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:01,524 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:01,624 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:01,724 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:01,824 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:01,924 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:02,025 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:02,125 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:02,225 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:02,325 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:02,425 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:02,526 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:02,626 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:02,726 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:02,826 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:02,927 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:03,027 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:03,127 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:03,227 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:03,327 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:03,427 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:03,528 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:03,628 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:03,728 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:03,828 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:03,928 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:04,029 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:04,129 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:04,229 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:04,329 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:04,430 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:04,530 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:04,630 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:04,730 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:04,830 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:04,931 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:05,031 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:05,131 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:05,231 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:05,331 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:05,432 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:05,532 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:05,632 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:05,732 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:05,832 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:05,933 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:06,033 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:06,133 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:06,233 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:06,333 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:06,434 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:06,534 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:06,634 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:06,734 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:06,834 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:06,935 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:07,035 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:07,135 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:07,235 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:07,335 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:07,436 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:07,536 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:07,636 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:07,736 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:07,837 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:07,937 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:08,037 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:08,137 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:08,237 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:08,338 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:08,438 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:08,538 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:08,638 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:08,738 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:08,839 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:08,939 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:09,039 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:09,139 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:09,239 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:09,340 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:09,440 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:09,540 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:09,592 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37903] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-05-10 04:55:09,640 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:09,693 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37903] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-05-10 04:55:09,740 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:09,840 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:09,894 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37903] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-05-10 04:55:09,941 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:10,041 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:10,141 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:10,195 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37903] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-05-10 04:55:10,241 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:10,342 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:10,442 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:10,542 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:10,642 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:10,695 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37903] master.MasterRpcServices(1227): Checking to see if procedure is done pid=88 2023-05-10 04:55:10,742 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:10,843 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:10,943 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:11,043 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:11,143 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:11,243 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:11,344 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:11,444 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:11,544 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:11,644 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:11,697 DEBUG [Listener at localhost/45007] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:11,698 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(1029): Stored pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=d2b34babccb97d3c1ac066d9e90ae654, daughterA=0141d5003cd2aac0ddef526f1a98709b, daughterB=b919ef6ce9c770ae26c6c5ee0d2ac459 2023-05-10 04:55:11,699 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=d2b34babccb97d3c1ac066d9e90ae654, daughterA=0141d5003cd2aac0ddef526f1a98709b, daughterB=b919ef6ce9c770ae26c6c5ee0d2ac459 2023-05-10 04:55:11,699 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=d2b34babccb97d3c1ac066d9e90ae654, daughterA=0141d5003cd2aac0ddef526f1a98709b, daughterB=b919ef6ce9c770ae26c6c5ee0d2ac459 2023-05-10 04:55:11,699 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=d2b34babccb97d3c1ac066d9e90ae654, daughterA=0141d5003cd2aac0ddef526f1a98709b, daughterB=b919ef6ce9c770ae26c6c5ee0d2ac459 2023-05-10 04:55:11,701 ERROR [PEWorker-1] assignment.SplitTableRegionProcedure(347): Splitting d2b34babccb97d3c1ac066d9e90ae654, pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=d2b34babccb97d3c1ac066d9e90ae654, daughterA=0141d5003cd2aac0ddef526f1a98709b, daughterB=b919ef6ce9c770ae26c6c5ee0d2ac459 org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,37995,1683694419499, table=testRollbackForSplitTableRegionWithReplica, region=bddd3bfc356670bc9cd9492c47d95a01 is currently in transition, give up at org.apache.hadoop.hbase.master.assignment.AssignmentManagerUtil.createUnassignProceduresForSplitOrMerge(AssignmentManagerUtil.java:120) at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.createUnassignProcedures(SplitTableRegionProcedure.java:881) at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.executeFromState(SplitTableRegionProcedure.java:298) at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.executeFromState(SplitTableRegionProcedure.java:94) at org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:188) at org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:922) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1646) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-10 04:55:11,703 INFO [PEWorker-1] procedure2.ProcedureExecutor(1528): Rolled back pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,37995,1683694419499, table=testRollbackForSplitTableRegionWithReplica, region=bddd3bfc356670bc9cd9492c47d95a01 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=d2b34babccb97d3c1ac066d9e90ae654, daughterA=0141d5003cd2aac0ddef526f1a98709b, daughterB=b919ef6ce9c770ae26c6c5ee0d2ac459 exec-time=6 msec 2023-05-10 04:55:11,744 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:11,845 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:11,945 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-05-10 04:55:12,045 INFO [StoreOpener-bddd3bfc356670bc9cd9492c47d95a01-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region bddd3bfc356670bc9cd9492c47d95a01 2023-05-10 04:55:12,048 DEBUG [StoreOpener-bddd3bfc356670bc9cd9492c47d95a01-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackForSplitTableRegionWithReplica/d2b34babccb97d3c1ac066d9e90ae654/cf1 2023-05-10 04:55:12,048 DEBUG [StoreOpener-bddd3bfc356670bc9cd9492c47d95a01-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackForSplitTableRegionWithReplica/d2b34babccb97d3c1ac066d9e90ae654/cf1 2023-05-10 04:55:12,049 INFO [StoreOpener-bddd3bfc356670bc9cd9492c47d95a01-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region bddd3bfc356670bc9cd9492c47d95a01 columnFamilyName cf1 2023-05-10 04:55:12,049 INFO [StoreOpener-bddd3bfc356670bc9cd9492c47d95a01-1] regionserver.HStore(310): Store=bddd3bfc356670bc9cd9492c47d95a01/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-10 04:55:12,051 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened bddd3bfc356670bc9cd9492c47d95a01; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72861446, jitterRate=0.08572015166282654}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-05-10 04:55:12,051 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for bddd3bfc356670bc9cd9492c47d95a01: 2023-05-10 04:55:12,051 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2334): Post open deploy tasks for testRollbackForSplitTableRegionWithReplica,,1683694448968_0001.bddd3bfc356670bc9cd9492c47d95a01., pid=94, masterSystemTime=1683694450314 2023-05-10 04:55:12,053 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2361): Finished post open deploy task for testRollbackForSplitTableRegionWithReplica,,1683694448968_0001.bddd3bfc356670bc9cd9492c47d95a01. 2023-05-10 04:55:12,053 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testRollbackForSplitTableRegionWithReplica,,1683694448968_0001.bddd3bfc356670bc9cd9492c47d95a01. 2023-05-10 04:55:12,053 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=93 updating hbase:meta row=bddd3bfc356670bc9cd9492c47d95a01, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:12,054 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackForSplitTableRegionWithReplica,,1683694448968.d2b34babccb97d3c1ac066d9e90ae654.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1683694512053"},{"qualifier":"server_0001","vlen":31,"tag":[],"timestamp":"1683694512053"},{"qualifier":"serverstartcode_0001","vlen":8,"tag":[],"timestamp":"1683694512053"},{"qualifier":"seqnumDuringOpen_0001","vlen":8,"tag":[],"timestamp":"1683694512053"}]},"ts":"1683694512053"} 2023-05-10 04:55:12,056 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=94, resume processing ppid=93 2023-05-10 04:55:12,056 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=94, ppid=93, state=SUCCESS; OpenRegionProcedure bddd3bfc356670bc9cd9492c47d95a01, server=jenkins-hbase4.apache.org,37995,1683694419499 in 1 mins, 1.894 sec 2023-05-10 04:55:12,058 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=93, resume processing ppid=88 2023-05-10 04:55:12,058 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=93, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=bddd3bfc356670bc9cd9492c47d95a01, ASSIGN in 1 mins, 2.051 sec 2023-05-10 04:55:12,060 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica in 1 mins, 2.577 sec 2023-05-10 04:55:12,061 WARN [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] ipc.RpcServer(528): (responseTooSlow): {"call":"ModifyTable(org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$ModifyTableRequest)","starttimems":"1683694449476","responsesize":"2","method":"ModifyTable","param":"TODO: class org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$ModifyTableRequest","processingtimems":62584,"client":"172.31.14.131:39844","queuetimems":0,"class":"HMaster"} 2023-05-10 04:55:12,061 DEBUG [RPCClient-NioEventLoopGroup-6-2] ipc.NettyRpcDuplexHandler(156): Unknown callId: 633, skipping over this response of 3 bytes 2023-05-10 04:55:13,949 INFO [Listener at localhost/45007] procedure2.ProcedureTestingUtility(335): procId=95 exception: org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,37995,1683694419499, table=testRollbackForSplitTableRegionWithReplica, region=bddd3bfc356670bc9cd9492c47d95a01 is currently in transition, give up 2023-05-10 04:55:13,951 WARN [Listener at localhost/45007] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-05-10 04:55:13,952 INFO [Listener at localhost/45007] client.HBaseAdmin$15(890): Started disable of testRollbackForSplitTableRegionWithReplica 2023-05-10 04:55:13,952 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testRollbackForSplitTableRegionWithReplica 2023-05-10 04:55:13,953 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] procedure2.ProcedureExecutor(1029): Stored pid=96, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-10 04:55:13,955 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-05-10 04:55:13,956 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683694513956"}]},"ts":"1683694513956"} 2023-05-10 04:55:13,957 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=DISABLING in hbase:meta 2023-05-10 04:55:13,958 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testRollbackForSplitTableRegionWithReplica to state=DISABLING 2023-05-10 04:55:13,960 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=97, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=d2b34babccb97d3c1ac066d9e90ae654, UNASSIGN}, {pid=98, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=bddd3bfc356670bc9cd9492c47d95a01, UNASSIGN}] 2023-05-10 04:55:13,961 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=98, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=bddd3bfc356670bc9cd9492c47d95a01, UNASSIGN 2023-05-10 04:55:13,962 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=bddd3bfc356670bc9cd9492c47d95a01, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:13,962 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1683694448968.d2b34babccb97d3c1ac066d9e90ae654.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1683694513962"},{"qualifier":"sn_0001","vlen":45,"tag":[],"timestamp":"1683694513962"},{"qualifier":"state_0001","vlen":7,"tag":[],"timestamp":"1683694513962"}]},"ts":"1683694513962"} 2023-05-10 04:55:13,963 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=99, ppid=98, state=RUNNABLE; CloseRegionProcedure bddd3bfc356670bc9cd9492c47d95a01, server=jenkins-hbase4.apache.org,37995,1683694419499}] 2023-05-10 04:55:13,966 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=97, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=d2b34babccb97d3c1ac066d9e90ae654, UNASSIGN 2023-05-10 04:55:13,966 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=97 updating hbase:meta row=d2b34babccb97d3c1ac066d9e90ae654, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:13,966 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1683694448968.d2b34babccb97d3c1ac066d9e90ae654.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1683694513966"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683694513966"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683694513966"}]},"ts":"1683694513966"} 2023-05-10 04:55:13,968 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=100, ppid=97, state=RUNNABLE; CloseRegionProcedure d2b34babccb97d3c1ac066d9e90ae654, server=jenkins-hbase4.apache.org,37823,1683694419534}] 2023-05-10 04:55:14,056 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-05-10 04:55:14,115 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close bddd3bfc356670bc9cd9492c47d95a01 2023-05-10 04:55:14,116 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing bddd3bfc356670bc9cd9492c47d95a01, disabling compactions & flushes 2023-05-10 04:55:14,116 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testRollbackForSplitTableRegionWithReplica,,1683694448968_0001.bddd3bfc356670bc9cd9492c47d95a01. 2023-05-10 04:55:14,116 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1683694448968_0001.bddd3bfc356670bc9cd9492c47d95a01. 2023-05-10 04:55:14,116 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1683694448968_0001.bddd3bfc356670bc9cd9492c47d95a01. after waiting 0 ms 2023-05-10 04:55:14,117 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1683694448968_0001.bddd3bfc356670bc9cd9492c47d95a01. 2023-05-10 04:55:14,117 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-10 04:55:14,118 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testRollbackForSplitTableRegionWithReplica,,1683694448968_0001.bddd3bfc356670bc9cd9492c47d95a01. 2023-05-10 04:55:14,118 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for bddd3bfc356670bc9cd9492c47d95a01: 2023-05-10 04:55:14,119 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed bddd3bfc356670bc9cd9492c47d95a01 2023-05-10 04:55:14,120 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close d2b34babccb97d3c1ac066d9e90ae654 2023-05-10 04:55:14,120 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing d2b34babccb97d3c1ac066d9e90ae654, disabling compactions & flushes 2023-05-10 04:55:14,120 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=bddd3bfc356670bc9cd9492c47d95a01, regionState=CLOSED 2023-05-10 04:55:14,120 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testRollbackForSplitTableRegionWithReplica,,1683694448968.d2b34babccb97d3c1ac066d9e90ae654. 2023-05-10 04:55:14,120 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1683694448968.d2b34babccb97d3c1ac066d9e90ae654. 2023-05-10 04:55:14,121 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1683694448968.d2b34babccb97d3c1ac066d9e90ae654. after waiting 0 ms 2023-05-10 04:55:14,121 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1683694448968.d2b34babccb97d3c1ac066d9e90ae654.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1683694514120"},{"qualifier":"state_0001","vlen":6,"tag":[],"timestamp":"1683694514120"}]},"ts":"1683694514120"} 2023-05-10 04:55:14,121 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1683694448968.d2b34babccb97d3c1ac066d9e90ae654. 2023-05-10 04:55:14,124 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=99, resume processing ppid=98 2023-05-10 04:55:14,124 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=99, ppid=98, state=SUCCESS; CloseRegionProcedure bddd3bfc356670bc9cd9492c47d95a01, server=jenkins-hbase4.apache.org,37995,1683694419499 in 159 msec 2023-05-10 04:55:14,125 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=98, ppid=96, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=bddd3bfc356670bc9cd9492c47d95a01, UNASSIGN in 164 msec 2023-05-10 04:55:14,125 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRollbackForSplitTableRegionWithReplica/d2b34babccb97d3c1ac066d9e90ae654/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-05-10 04:55:14,126 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-10 04:55:14,126 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testRollbackForSplitTableRegionWithReplica,,1683694448968.d2b34babccb97d3c1ac066d9e90ae654. 2023-05-10 04:55:14,127 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for d2b34babccb97d3c1ac066d9e90ae654: 2023-05-10 04:55:14,128 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed d2b34babccb97d3c1ac066d9e90ae654 2023-05-10 04:55:14,128 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=97 updating hbase:meta row=d2b34babccb97d3c1ac066d9e90ae654, regionState=CLOSED 2023-05-10 04:55:14,128 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1683694448968.d2b34babccb97d3c1ac066d9e90ae654.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1683694514128"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683694514128"}]},"ts":"1683694514128"} 2023-05-10 04:55:14,130 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=100, resume processing ppid=97 2023-05-10 04:55:14,130 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=100, ppid=97, state=SUCCESS; CloseRegionProcedure d2b34babccb97d3c1ac066d9e90ae654, server=jenkins-hbase4.apache.org,37823,1683694419534 in 161 msec 2023-05-10 04:55:14,132 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=97, resume processing ppid=96 2023-05-10 04:55:14,132 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=97, ppid=96, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=d2b34babccb97d3c1ac066d9e90ae654, UNASSIGN in 170 msec 2023-05-10 04:55:14,132 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683694514132"}]},"ts":"1683694514132"} 2023-05-10 04:55:14,133 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=DISABLED in hbase:meta 2023-05-10 04:55:14,135 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testRollbackForSplitTableRegionWithReplica to state=DISABLED 2023-05-10 04:55:14,136 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica in 182 msec 2023-05-10 04:55:14,258 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(1227): Checking to see if procedure is done pid=96 2023-05-10 04:55:14,258 INFO [Listener at localhost/45007] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testRollbackForSplitTableRegionWithReplica, procId: 96 completed 2023-05-10 04:55:14,258 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete testRollbackForSplitTableRegionWithReplica 2023-05-10 04:55:14,259 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] procedure2.ProcedureExecutor(1029): Stored pid=101, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-10 04:55:14,261 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=101, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-10 04:55:14,261 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=101, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-10 04:55:14,261 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(1227): Checking to see if procedure is done pid=101 2023-05-10 04:55:14,266 DEBUG [HFileArchiver-9] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/d2b34babccb97d3c1ac066d9e90ae654 2023-05-10 04:55:14,267 DEBUG [HFileArchiver-9] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/d2b34babccb97d3c1ac066d9e90ae654/cf1, FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/d2b34babccb97d3c1ac066d9e90ae654/recovered.edits] 2023-05-10 04:55:14,272 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/d2b34babccb97d3c1ac066d9e90ae654/recovered.edits/7.seqid to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testRollbackForSplitTableRegionWithReplica/d2b34babccb97d3c1ac066d9e90ae654/recovered.edits/7.seqid 2023-05-10 04:55:14,272 DEBUG [HFileArchiver-9] backup.HFileArchiver(596): Deleted hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/d2b34babccb97d3c1ac066d9e90ae654 2023-05-10 04:55:14,273 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackForSplitTableRegionWithReplica regions 2023-05-10 04:55:14,275 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=101, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-10 04:55:14,277 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testRollbackForSplitTableRegionWithReplica from hbase:meta 2023-05-10 04:55:14,280 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testRollbackForSplitTableRegionWithReplica' descriptor. 2023-05-10 04:55:14,281 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=101, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-10 04:55:14,281 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testRollbackForSplitTableRegionWithReplica' from region states. 2023-05-10 04:55:14,282 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica,,1683694448968.d2b34babccb97d3c1ac066d9e90ae654.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1683694514282"}]},"ts":"9223372036854775807"} 2023-05-10 04:55:14,282 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica,,1683694448968_0001.bddd3bfc356670bc9cd9492c47d95a01.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1683694514282"}]},"ts":"9223372036854775807"} 2023-05-10 04:55:14,283 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 2 regions from META 2023-05-10 04:55:14,283 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => d2b34babccb97d3c1ac066d9e90ae654, NAME => 'testRollbackForSplitTableRegionWithReplica,,1683694448968.d2b34babccb97d3c1ac066d9e90ae654.', STARTKEY => '', ENDKEY => ''}, {ENCODED => bddd3bfc356670bc9cd9492c47d95a01, NAME => 'testRollbackForSplitTableRegionWithReplica,,1683694448968_0001.bddd3bfc356670bc9cd9492c47d95a01.', STARTKEY => '', ENDKEY => '', REPLICA_ID => 1}] 2023-05-10 04:55:14,283 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testRollbackForSplitTableRegionWithReplica' as deleted. 2023-05-10 04:55:14,284 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1683694514283"}]},"ts":"9223372036854775807"} 2023-05-10 04:55:14,285 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testRollbackForSplitTableRegionWithReplica state from META 2023-05-10 04:55:14,287 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=101, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-10 04:55:14,287 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica in 28 msec 2023-05-10 04:55:14,363 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(1227): Checking to see if procedure is done pid=101 2023-05-10 04:55:14,363 INFO [Listener at localhost/45007] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testRollbackForSplitTableRegionWithReplica, procId: 101 completed 2023-05-10 04:55:14,377 INFO [Listener at localhost/45007] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testRollbackForSplitTableRegionWithReplica Thread=347 (was 412), OpenFileDescriptor=658 (was 868), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=175 (was 323), ProcessCount=172 (was 175), AvailableMemoryMB=9557 (was 9626) 2023-05-10 04:55:14,389 INFO [Listener at localhost/45007] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testInvalidSplitKey Thread=348, OpenFileDescriptor=658, MaxFileDescriptor=60000, SystemLoadAverage=175, ProcessCount=172, AvailableMemoryMB=9556 2023-05-10 04:55:14,390 WARN [Listener at localhost/45007] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-05-10 04:55:14,392 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-05-10 04:55:14,393 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-05-10 04:55:14,394 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(1029): Stored pid=102, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testInvalidSplitKey 2023-05-10 04:55:14,395 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_PRE_OPERATION 2023-05-10 04:55:14,396 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-05-10 04:55:14,398 DEBUG [HFileArchiver-10] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testInvalidSplitKey/28e42ae67401b1d1f2840cbe891ccf89 2023-05-10 04:55:14,398 DEBUG [HFileArchiver-10] backup.HFileArchiver(153): Directory hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testInvalidSplitKey/28e42ae67401b1d1f2840cbe891ccf89 empty. 2023-05-10 04:55:14,398 DEBUG [HFileArchiver-10] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testInvalidSplitKey/28e42ae67401b1d1f2840cbe891ccf89 2023-05-10 04:55:14,398 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testInvalidSplitKey regions 2023-05-10 04:55:14,410 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testInvalidSplitKey/.tabledesc/.tableinfo.0000000001 2023-05-10 04:55:14,411 INFO [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(7675): creating {ENCODED => 28e42ae67401b1d1f2840cbe891ccf89, NAME => 'testInvalidSplitKey,,1683694514393.28e42ae67401b1d1f2840cbe891ccf89.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testInvalidSplitKey', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp 2023-05-10 04:55:14,419 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(866): Instantiated testInvalidSplitKey,,1683694514393.28e42ae67401b1d1f2840cbe891ccf89.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-10 04:55:14,419 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1604): Closing 28e42ae67401b1d1f2840cbe891ccf89, disabling compactions & flushes 2023-05-10 04:55:14,419 INFO [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1626): Closing region testInvalidSplitKey,,1683694514393.28e42ae67401b1d1f2840cbe891ccf89. 2023-05-10 04:55:14,419 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testInvalidSplitKey,,1683694514393.28e42ae67401b1d1f2840cbe891ccf89. 2023-05-10 04:55:14,419 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1714): Acquired close lock on testInvalidSplitKey,,1683694514393.28e42ae67401b1d1f2840cbe891ccf89. after waiting 0 ms 2023-05-10 04:55:14,419 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1724): Updates disabled for region testInvalidSplitKey,,1683694514393.28e42ae67401b1d1f2840cbe891ccf89. 2023-05-10 04:55:14,420 INFO [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1838): Closed testInvalidSplitKey,,1683694514393.28e42ae67401b1d1f2840cbe891ccf89. 2023-05-10 04:55:14,420 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1558): Region close journal for 28e42ae67401b1d1f2840cbe891ccf89: 2023-05-10 04:55:14,421 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_ADD_TO_META 2023-05-10 04:55:14,422 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testInvalidSplitKey,,1683694514393.28e42ae67401b1d1f2840cbe891ccf89.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1683694514422"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683694514422"}]},"ts":"1683694514422"} 2023-05-10 04:55:14,424 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-05-10 04:55:14,424 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-05-10 04:55:14,424 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683694514424"}]},"ts":"1683694514424"} 2023-05-10 04:55:14,425 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=ENABLING in hbase:meta 2023-05-10 04:55:14,428 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-05-10 04:55:14,429 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-05-10 04:55:14,429 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-05-10 04:55:14,429 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-05-10 04:55:14,429 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-05-10 04:55:14,430 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=103, ppid=102, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testInvalidSplitKey, region=28e42ae67401b1d1f2840cbe891ccf89, ASSIGN}] 2023-05-10 04:55:14,431 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=103, ppid=102, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testInvalidSplitKey, region=28e42ae67401b1d1f2840cbe891ccf89, ASSIGN 2023-05-10 04:55:14,431 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=103, ppid=102, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testInvalidSplitKey, region=28e42ae67401b1d1f2840cbe891ccf89, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,37823,1683694419534; forceNewPlan=false, retain=false 2023-05-10 04:55:14,582 INFO [jenkins-hbase4:37903] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-05-10 04:55:14,583 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=103 updating hbase:meta row=28e42ae67401b1d1f2840cbe891ccf89, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:14,583 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testInvalidSplitKey,,1683694514393.28e42ae67401b1d1f2840cbe891ccf89.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1683694514583"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683694514583"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683694514583"}]},"ts":"1683694514583"} 2023-05-10 04:55:14,584 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=104, ppid=103, state=RUNNABLE; OpenRegionProcedure 28e42ae67401b1d1f2840cbe891ccf89, server=jenkins-hbase4.apache.org,37823,1683694419534}] 2023-05-10 04:55:14,739 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testInvalidSplitKey,,1683694514393.28e42ae67401b1d1f2840cbe891ccf89. 2023-05-10 04:55:14,739 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 28e42ae67401b1d1f2840cbe891ccf89, NAME => 'testInvalidSplitKey,,1683694514393.28e42ae67401b1d1f2840cbe891ccf89.', STARTKEY => '', ENDKEY => ''} 2023-05-10 04:55:14,739 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-05-10 04:55:14,740 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testInvalidSplitKey 28e42ae67401b1d1f2840cbe891ccf89 2023-05-10 04:55:14,740 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testInvalidSplitKey,,1683694514393.28e42ae67401b1d1f2840cbe891ccf89.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-10 04:55:14,740 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 28e42ae67401b1d1f2840cbe891ccf89 2023-05-10 04:55:14,740 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 28e42ae67401b1d1f2840cbe891ccf89 2023-05-10 04:55:14,741 INFO [StoreOpener-28e42ae67401b1d1f2840cbe891ccf89-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 28e42ae67401b1d1f2840cbe891ccf89 2023-05-10 04:55:14,742 DEBUG [StoreOpener-28e42ae67401b1d1f2840cbe891ccf89-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testInvalidSplitKey/28e42ae67401b1d1f2840cbe891ccf89/cf1 2023-05-10 04:55:14,742 DEBUG [StoreOpener-28e42ae67401b1d1f2840cbe891ccf89-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testInvalidSplitKey/28e42ae67401b1d1f2840cbe891ccf89/cf1 2023-05-10 04:55:14,743 INFO [StoreOpener-28e42ae67401b1d1f2840cbe891ccf89-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 28e42ae67401b1d1f2840cbe891ccf89 columnFamilyName cf1 2023-05-10 04:55:14,743 INFO [StoreOpener-28e42ae67401b1d1f2840cbe891ccf89-1] regionserver.HStore(310): Store=28e42ae67401b1d1f2840cbe891ccf89/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-10 04:55:14,743 INFO [StoreOpener-28e42ae67401b1d1f2840cbe891ccf89-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 28e42ae67401b1d1f2840cbe891ccf89 2023-05-10 04:55:14,744 DEBUG [StoreOpener-28e42ae67401b1d1f2840cbe891ccf89-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testInvalidSplitKey/28e42ae67401b1d1f2840cbe891ccf89/cf2 2023-05-10 04:55:14,744 DEBUG [StoreOpener-28e42ae67401b1d1f2840cbe891ccf89-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testInvalidSplitKey/28e42ae67401b1d1f2840cbe891ccf89/cf2 2023-05-10 04:55:14,745 INFO [StoreOpener-28e42ae67401b1d1f2840cbe891ccf89-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 28e42ae67401b1d1f2840cbe891ccf89 columnFamilyName cf2 2023-05-10 04:55:14,745 INFO [StoreOpener-28e42ae67401b1d1f2840cbe891ccf89-1] regionserver.HStore(310): Store=28e42ae67401b1d1f2840cbe891ccf89/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-10 04:55:14,746 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testInvalidSplitKey/28e42ae67401b1d1f2840cbe891ccf89 2023-05-10 04:55:14,746 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testInvalidSplitKey/28e42ae67401b1d1f2840cbe891ccf89 2023-05-10 04:55:14,747 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testInvalidSplitKey descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-05-10 04:55:14,748 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 28e42ae67401b1d1f2840cbe891ccf89 2023-05-10 04:55:14,750 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testInvalidSplitKey/28e42ae67401b1d1f2840cbe891ccf89/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-05-10 04:55:14,750 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 28e42ae67401b1d1f2840cbe891ccf89; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72783191, jitterRate=0.08455406129360199}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-05-10 04:55:14,750 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 28e42ae67401b1d1f2840cbe891ccf89: 2023-05-10 04:55:14,751 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2334): Post open deploy tasks for testInvalidSplitKey,,1683694514393.28e42ae67401b1d1f2840cbe891ccf89., pid=104, masterSystemTime=1683694514736 2023-05-10 04:55:14,752 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2361): Finished post open deploy task for testInvalidSplitKey,,1683694514393.28e42ae67401b1d1f2840cbe891ccf89. 2023-05-10 04:55:14,752 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testInvalidSplitKey,,1683694514393.28e42ae67401b1d1f2840cbe891ccf89. 2023-05-10 04:55:14,752 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=103 updating hbase:meta row=28e42ae67401b1d1f2840cbe891ccf89, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:14,752 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testInvalidSplitKey,,1683694514393.28e42ae67401b1d1f2840cbe891ccf89.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1683694514752"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1683694514752"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1683694514752"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683694514752"}]},"ts":"1683694514752"} 2023-05-10 04:55:14,755 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=104, resume processing ppid=103 2023-05-10 04:55:14,755 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=104, ppid=103, state=SUCCESS; OpenRegionProcedure 28e42ae67401b1d1f2840cbe891ccf89, server=jenkins-hbase4.apache.org,37823,1683694419534 in 170 msec 2023-05-10 04:55:14,757 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=103, resume processing ppid=102 2023-05-10 04:55:14,757 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=103, ppid=102, state=SUCCESS; TransitRegionStateProcedure table=testInvalidSplitKey, region=28e42ae67401b1d1f2840cbe891ccf89, ASSIGN in 325 msec 2023-05-10 04:55:14,757 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-05-10 04:55:14,757 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683694514757"}]},"ts":"1683694514757"} 2023-05-10 04:55:14,758 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=ENABLED in hbase:meta 2023-05-10 04:55:14,760 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_POST_OPERATION 2023-05-10 04:55:14,762 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey in 367 msec 2023-05-10 04:55:14,902 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-10 04:55:14,903 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-05-10 04:55:14,903 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-05-10 04:55:14,903 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:55:14,903 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-05-10 04:55:14,903 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-05-10 04:55:14,904 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-10 04:55:14,904 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-10 04:55:14,906 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:14,906 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:14,906 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:14,906 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:14,906 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:14,906 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:14,906 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:14,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:14,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:14,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:14,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-10 04:55:14,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-10 04:55:14,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-10 04:55:14,907 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:14,907 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-10 04:55:14,907 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-10 04:55:14,907 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-10 04:55:14,907 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-10 04:55:14,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:55:14,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-10 04:55:14,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:55:14,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:55:14,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-10 04:55:14,907 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-05-10 04:55:14,907 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-05-10 04:55:14,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-10 04:55:14,908 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-05-10 04:55:14,908 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-05-10 04:55:14,908 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-05-10 04:55:14,908 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:55:14,908 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-05-10 04:55:14,908 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-05-10 04:55:14,908 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-05-10 04:55:14,908 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:55:14,908 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-05-10 04:55:14,908 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-05-10 04:55:14,908 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:55:14,908 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-05-10 04:55:14,908 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-05-10 04:55:14,908 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-05-10 04:55:14,908 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37995,1683694419499' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-05-10 04:55:14,908 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-05-10 04:55:14,908 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-10 04:55:14,908 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1683694514393.28e42ae67401b1d1f2840cbe891ccf89. 2023-05-10 04:55:14,908 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36665,1683694419452' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-05-10 04:55:14,908 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1683694514393.28e42ae67401b1d1f2840cbe891ccf89. started... 2023-05-10 04:55:14,908 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-10 04:55:14,909 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 28e42ae67401b1d1f2840cbe891ccf89 2/2 column families, dataSize=120 B heapSize=928 B 2023-05-10 04:55:14,910 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-10 04:55:14,911 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-10 04:55:14,911 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-10 04:55:14,911 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:55:14,911 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-10 04:55:14,911 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:55:14,920 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testInvalidSplitKey/28e42ae67401b1d1f2840cbe891ccf89/.tmp/cf1/ec297150d49e422baece5d4186fed525 2023-05-10 04:55:14,936 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testInvalidSplitKey/28e42ae67401b1d1f2840cbe891ccf89/.tmp/cf2/5ff7d2b62aa0473a8aa0ceeda965c0c0 2023-05-10 04:55:14,941 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testInvalidSplitKey/28e42ae67401b1d1f2840cbe891ccf89/.tmp/cf1/ec297150d49e422baece5d4186fed525 as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testInvalidSplitKey/28e42ae67401b1d1f2840cbe891ccf89/cf1/ec297150d49e422baece5d4186fed525 2023-05-10 04:55:14,945 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testInvalidSplitKey/28e42ae67401b1d1f2840cbe891ccf89/cf1/ec297150d49e422baece5d4186fed525, entries=2, sequenceid=6, filesize=4.8 K 2023-05-10 04:55:14,946 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testInvalidSplitKey/28e42ae67401b1d1f2840cbe891ccf89/.tmp/cf2/5ff7d2b62aa0473a8aa0ceeda965c0c0 as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testInvalidSplitKey/28e42ae67401b1d1f2840cbe891ccf89/cf2/5ff7d2b62aa0473a8aa0ceeda965c0c0 2023-05-10 04:55:14,950 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testInvalidSplitKey/28e42ae67401b1d1f2840cbe891ccf89/cf2/5ff7d2b62aa0473a8aa0ceeda965c0c0, entries=2, sequenceid=6, filesize=4.8 K 2023-05-10 04:55:14,951 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 28e42ae67401b1d1f2840cbe891ccf89 in 42ms, sequenceid=6, compaction requested=false 2023-05-10 04:55:14,951 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testInvalidSplitKey' 2023-05-10 04:55:14,952 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 28e42ae67401b1d1f2840cbe891ccf89: 2023-05-10 04:55:14,952 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1683694514393.28e42ae67401b1d1f2840cbe891ccf89. 2023-05-10 04:55:14,952 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-10 04:55:14,952 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-10 04:55:14,952 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:14,952 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-05-10 04:55:14,952 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37823,1683694419534' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-05-10 04:55:14,954 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-10 04:55:14,954 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:14,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:14,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:55:14,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:55:14,954 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-10 04:55:14,954 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:55:14,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:55:14,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:55:14,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-10 04:55:14,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:14,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:14,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:14,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:55:14,957 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37823,1683694419534' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-05-10 04:55:14,957 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4ae1d92c[Count = 0] remaining members to acquire global barrier 2023-05-10 04:55:14,957 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-05-10 04:55:14,957 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-10 04:55:14,959 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-10 04:55:14,959 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-10 04:55:14,959 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-10 04:55:14,959 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-10 04:55:14,959 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-10 04:55:14,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-10 04:55:14,959 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-10 04:55:14,959 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-05-10 04:55:14,959 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:14,960 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-10 04:55:14,959 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-05-10 04:55:14,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-10 04:55:14,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-10 04:55:14,960 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-05-10 04:55:14,960 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,36665,1683694419452' in zk 2023-05-10 04:55:14,960 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-05-10 04:55:14,960 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,37823,1683694419534' in zk 2023-05-10 04:55:14,960 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-05-10 04:55:14,960 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-05-10 04:55:14,960 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,37995,1683694419499' in zk 2023-05-10 04:55:14,962 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-05-10 04:55:14,962 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:55:14,962 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-05-10 04:55:14,962 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-05-10 04:55:14,962 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-05-10 04:55:14,962 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:14,962 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:55:14,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:14,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:55:14,963 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:55:14,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:55:14,963 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-05-10 04:55:14,963 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-05-10 04:55:14,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:55:14,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:55:14,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-10 04:55:14,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:14,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:14,964 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:14,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:55:14,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-10 04:55:14,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:14,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:14,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:14,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,37823,1683694419534': 2023-05-10 04:55:14,966 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37823,1683694419534' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-05-10 04:55:14,966 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-05-10 04:55:14,966 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-10 04:55:14,966 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-10 04:55:14,966 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-05-10 04:55:14,966 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-10 04:55:14,968 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-10 04:55:14,968 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-10 04:55:14,968 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-10 04:55:14,968 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-10 04:55:14,968 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-10 04:55:14,968 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:14,968 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-10 04:55:14,968 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:14,968 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:14,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-10 04:55:14,968 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-10 04:55:14,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-10 04:55:14,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:55:14,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:55:14,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-10 04:55:14,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-10 04:55:14,969 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:14,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:14,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:55:14,969 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:14,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-10 04:55:14,969 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:14,969 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:14,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:14,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:14,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-10 04:55:14,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-10 04:55:14,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-10 04:55:14,970 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:14,970 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:55:14,971 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:14,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-10 04:55:14,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:14,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:14,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:14,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:55:14,972 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:14,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-10 04:55:14,972 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:14,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:14,973 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:14,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:14,973 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:14,977 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:14,977 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-10 04:55:14,977 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:14,977 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:14,977 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:14,977 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:14,977 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-05-10 04:55:14,977 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:14,977 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-10 04:55:14,977 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:14,977 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:14,977 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:55:14,977 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:14,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:14,978 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-05-10 04:55:14,977 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:14,978 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-10 04:55:14,977 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:14,977 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:14,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:14,977 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:14,978 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:14,978 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-10 04:55:14,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:14,978 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:14,978 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:14,978 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-10 04:55:14,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:14,978 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:14,978 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:14,978 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:14,978 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:14,979 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-10 04:55:14,979 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-10 04:55:15,078 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-10 04:55:15,079 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-10 04:55:15,092 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-10 04:55:15,094 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-05-10 04:55:15,094 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-05-10 04:55:15,094 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-05-10 04:55:15,094 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:55:15,094 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-05-10 04:55:15,094 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-05-10 04:55:15,094 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-10 04:55:15,094 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-10 04:55:15,095 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:15,095 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:15,095 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:15,096 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:15,096 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:15,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:15,095 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:15,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:15,095 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:15,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:15,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-10 04:55:15,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-10 04:55:15,096 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:15,096 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-10 04:55:15,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-10 04:55:15,096 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-10 04:55:15,096 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-10 04:55:15,097 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-10 04:55:15,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:55:15,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-10 04:55:15,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:55:15,097 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-05-10 04:55:15,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-10 04:55:15,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:55:15,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-10 04:55:15,097 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-05-10 04:55:15,097 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-05-10 04:55:15,097 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-05-10 04:55:15,097 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-05-10 04:55:15,097 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-05-10 04:55:15,097 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-05-10 04:55:15,097 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-05-10 04:55:15,097 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-05-10 04:55:15,097 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:55:15,098 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-05-10 04:55:15,098 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-05-10 04:55:15,098 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-05-10 04:55:15,098 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:55:15,098 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:55:15,098 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-05-10 04:55:15,098 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-05-10 04:55:15,098 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36665,1683694419452' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-05-10 04:55:15,098 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-05-10 04:55:15,098 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-05-10 04:55:15,098 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-05-10 04:55:15,098 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37995,1683694419499' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-05-10 04:55:15,098 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-10 04:55:15,098 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-10 04:55:15,098 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1683694514393.28e42ae67401b1d1f2840cbe891ccf89. 2023-05-10 04:55:15,098 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1683694514393.28e42ae67401b1d1f2840cbe891ccf89. started... 2023-05-10 04:55:15,098 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 28e42ae67401b1d1f2840cbe891ccf89 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-10 04:55:15,102 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-10 04:55:15,102 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-10 04:55:15,102 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-10 04:55:15,102 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-10 04:55:15,102 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:55:15,102 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:55:15,108 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testInvalidSplitKey/28e42ae67401b1d1f2840cbe891ccf89/.tmp/cf1/42f3469d25b14586ad2d722e4281417a 2023-05-10 04:55:15,118 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testInvalidSplitKey/28e42ae67401b1d1f2840cbe891ccf89/.tmp/cf2/1df608e0fef74a0b994e80ed5c6bf96c 2023-05-10 04:55:15,122 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testInvalidSplitKey/28e42ae67401b1d1f2840cbe891ccf89/.tmp/cf1/42f3469d25b14586ad2d722e4281417a as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testInvalidSplitKey/28e42ae67401b1d1f2840cbe891ccf89/cf1/42f3469d25b14586ad2d722e4281417a 2023-05-10 04:55:15,126 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testInvalidSplitKey/28e42ae67401b1d1f2840cbe891ccf89/cf1/42f3469d25b14586ad2d722e4281417a, entries=10, sequenceid=19, filesize=5.0 K 2023-05-10 04:55:15,126 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testInvalidSplitKey/28e42ae67401b1d1f2840cbe891ccf89/.tmp/cf2/1df608e0fef74a0b994e80ed5c6bf96c as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testInvalidSplitKey/28e42ae67401b1d1f2840cbe891ccf89/cf2/1df608e0fef74a0b994e80ed5c6bf96c 2023-05-10 04:55:15,129 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testInvalidSplitKey/28e42ae67401b1d1f2840cbe891ccf89/cf2/1df608e0fef74a0b994e80ed5c6bf96c, entries=10, sequenceid=19, filesize=5.0 K 2023-05-10 04:55:15,130 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 28e42ae67401b1d1f2840cbe891ccf89 in 32ms, sequenceid=19, compaction requested=false 2023-05-10 04:55:15,130 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 28e42ae67401b1d1f2840cbe891ccf89: 2023-05-10 04:55:15,130 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1683694514393.28e42ae67401b1d1f2840cbe891ccf89. 2023-05-10 04:55:15,130 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-10 04:55:15,130 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-10 04:55:15,131 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:15,131 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-05-10 04:55:15,131 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37823,1683694419534' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-05-10 04:55:15,133 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:15,133 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-10 04:55:15,133 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:15,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:55:15,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:55:15,134 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-10 04:55:15,134 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:55:15,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:55:15,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:55:15,134 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-10 04:55:15,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:15,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:15,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:15,135 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:55:15,136 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37823,1683694419534' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-05-10 04:55:15,136 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3fdcde54[Count = 0] remaining members to acquire global barrier 2023-05-10 04:55:15,136 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-05-10 04:55:15,136 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-10 04:55:15,137 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-10 04:55:15,137 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-10 04:55:15,137 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-10 04:55:15,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-10 04:55:15,137 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-10 04:55:15,137 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-05-10 04:55:15,137 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-10 04:55:15,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-10 04:55:15,137 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-05-10 04:55:15,137 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-10 04:55:15,137 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,37995,1683694419499' in zk 2023-05-10 04:55:15,137 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-05-10 04:55:15,137 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-05-10 04:55:15,137 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-10 04:55:15,137 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,37823,1683694419534' in zk 2023-05-10 04:55:15,137 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-05-10 04:55:15,137 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-05-10 04:55:15,137 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,36665,1683694419452' in zk 2023-05-10 04:55:15,137 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:15,138 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-10 04:55:15,138 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-05-10 04:55:15,138 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:55:15,138 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-05-10 04:55:15,139 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-05-10 04:55:15,139 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:15,139 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:55:15,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:15,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:55:15,139 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:55:15,139 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-05-10 04:55:15,141 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-05-10 04:55:15,141 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:55:15,141 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:55:15,141 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-05-10 04:55:15,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:55:15,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-10 04:55:15,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:15,142 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:15,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:15,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:55:15,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-10 04:55:15,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:15,143 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:15,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:15,144 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,37823,1683694419534': 2023-05-10 04:55:15,144 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37823,1683694419534' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-05-10 04:55:15,144 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-05-10 04:55:15,144 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-10 04:55:15,144 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-10 04:55:15,144 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-05-10 04:55:15,144 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-10 04:55:15,146 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-10 04:55:15,146 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-10 04:55:15,146 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-10 04:55:15,146 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-10 04:55:15,146 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:15,146 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-10 04:55:15,146 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:15,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-10 04:55:15,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:55:15,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:55:15,146 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-10 04:55:15,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-10 04:55:15,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-10 04:55:15,146 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-10 04:55:15,146 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:15,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:55:15,146 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:15,146 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:15,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-10 04:55:15,146 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:15,146 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:15,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:15,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-10 04:55:15,147 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:15,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-10 04:55:15,147 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:15,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:15,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:55:15,147 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:15,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-10 04:55:15,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-10 04:55:15,147 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-10 04:55:15,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:15,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:15,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:15,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:55:15,148 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:15,148 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-10 04:55:15,149 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:15,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:15,149 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:15,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:15,149 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:15,151 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:15,151 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-10 04:55:15,151 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:15,151 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:15,151 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-10 04:55:15,151 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:15,151 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:15,151 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:15,151 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:15,152 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-10 04:55:15,152 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:15,152 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:15,152 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:55:15,152 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:15,151 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:15,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:15,151 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:15,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:15,151 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:15,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:15,152 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-05-10 04:55:15,152 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-10 04:55:15,152 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:15,152 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:15,152 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:15,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:15,152 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:15,152 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:15,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:15,152 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-10 04:55:15,152 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-10 04:55:15,152 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-05-10 04:55:15,152 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:15,153 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-10 04:55:15,253 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-10 04:55:15,254 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-10 04:55:15,265 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-10 04:55:15,266 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-05-10 04:55:15,266 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-05-10 04:55:15,266 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-05-10 04:55:15,266 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:55:15,266 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-05-10 04:55:15,266 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-05-10 04:55:15,267 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-10 04:55:15,267 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-10 04:55:15,268 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:15,268 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:15,268 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:15,268 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:15,268 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:15,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:15,268 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:15,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:15,268 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:15,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:15,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-10 04:55:15,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-10 04:55:15,269 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:15,269 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-10 04:55:15,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-10 04:55:15,269 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-10 04:55:15,269 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-10 04:55:15,269 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-10 04:55:15,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:55:15,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:55:15,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-10 04:55:15,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-10 04:55:15,270 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-05-10 04:55:15,270 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-05-10 04:55:15,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:55:15,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-10 04:55:15,270 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-05-10 04:55:15,270 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-05-10 04:55:15,270 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-05-10 04:55:15,270 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-05-10 04:55:15,271 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-05-10 04:55:15,271 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-05-10 04:55:15,271 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-05-10 04:55:15,271 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-05-10 04:55:15,271 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-05-10 04:55:15,271 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-05-10 04:55:15,271 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:55:15,271 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:55:15,271 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-05-10 04:55:15,271 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-05-10 04:55:15,271 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-05-10 04:55:15,271 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37995,1683694419499' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-05-10 04:55:15,271 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:55:15,271 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-05-10 04:55:15,271 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-10 04:55:15,271 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-10 04:55:15,271 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1683694514393.28e42ae67401b1d1f2840cbe891ccf89. 2023-05-10 04:55:15,271 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1683694514393.28e42ae67401b1d1f2840cbe891ccf89. started... 2023-05-10 04:55:15,271 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-05-10 04:55:15,271 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36665,1683694419452' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-05-10 04:55:15,271 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 28e42ae67401b1d1f2840cbe891ccf89 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-10 04:55:15,273 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-10 04:55:15,274 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-10 04:55:15,274 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:55:15,274 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-10 04:55:15,275 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-10 04:55:15,275 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:55:15,280 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testInvalidSplitKey/28e42ae67401b1d1f2840cbe891ccf89/.tmp/cf1/8c06e6af40d343b890bafa9b20b78e1e 2023-05-10 04:55:15,291 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testInvalidSplitKey/28e42ae67401b1d1f2840cbe891ccf89/.tmp/cf2/667e01382a094633be211ae11ba65d04 2023-05-10 04:55:15,295 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testInvalidSplitKey/28e42ae67401b1d1f2840cbe891ccf89/.tmp/cf1/8c06e6af40d343b890bafa9b20b78e1e as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testInvalidSplitKey/28e42ae67401b1d1f2840cbe891ccf89/cf1/8c06e6af40d343b890bafa9b20b78e1e 2023-05-10 04:55:15,298 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testInvalidSplitKey/28e42ae67401b1d1f2840cbe891ccf89/cf1/8c06e6af40d343b890bafa9b20b78e1e, entries=10, sequenceid=32, filesize=5.0 K 2023-05-10 04:55:15,299 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testInvalidSplitKey/28e42ae67401b1d1f2840cbe891ccf89/.tmp/cf2/667e01382a094633be211ae11ba65d04 as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testInvalidSplitKey/28e42ae67401b1d1f2840cbe891ccf89/cf2/667e01382a094633be211ae11ba65d04 2023-05-10 04:55:15,303 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testInvalidSplitKey/28e42ae67401b1d1f2840cbe891ccf89/cf2/667e01382a094633be211ae11ba65d04, entries=10, sequenceid=32, filesize=5.0 K 2023-05-10 04:55:15,303 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 28e42ae67401b1d1f2840cbe891ccf89 in 32ms, sequenceid=32, compaction requested=true 2023-05-10 04:55:15,304 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 28e42ae67401b1d1f2840cbe891ccf89: 2023-05-10 04:55:15,304 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1683694514393.28e42ae67401b1d1f2840cbe891ccf89. 2023-05-10 04:55:15,304 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-10 04:55:15,304 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-10 04:55:15,304 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:15,304 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-05-10 04:55:15,304 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37823,1683694419534' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-05-10 04:55:15,305 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-10 04:55:15,305 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:15,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:15,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:55:15,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:55:15,306 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-10 04:55:15,306 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:55:15,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:55:15,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:55:15,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-10 04:55:15,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:15,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:15,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:15,307 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:55:15,307 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37823,1683694419534' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-05-10 04:55:15,307 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@64fef262[Count = 0] remaining members to acquire global barrier 2023-05-10 04:55:15,307 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-05-10 04:55:15,308 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-10 04:55:15,309 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-10 04:55:15,309 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-10 04:55:15,309 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-10 04:55:15,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-10 04:55:15,309 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-10 04:55:15,309 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-05-10 04:55:15,309 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-05-10 04:55:15,309 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,37995,1683694419499' in zk 2023-05-10 04:55:15,309 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-10 04:55:15,309 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:15,309 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-10 04:55:15,309 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-10 04:55:15,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-10 04:55:15,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-10 04:55:15,309 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-05-10 04:55:15,309 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-05-10 04:55:15,309 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-05-10 04:55:15,309 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,37823,1683694419534' in zk 2023-05-10 04:55:15,309 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-05-10 04:55:15,309 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,36665,1683694419452' in zk 2023-05-10 04:55:15,311 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-05-10 04:55:15,311 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:55:15,311 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-05-10 04:55:15,312 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-05-10 04:55:15,312 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-05-10 04:55:15,312 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:55:15,312 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:15,312 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-05-10 04:55:15,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:15,312 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:55:15,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:55:15,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:55:15,312 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-05-10 04:55:15,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:55:15,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:55:15,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-10 04:55:15,313 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:15,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:15,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:15,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:55:15,314 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-10 04:55:15,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:15,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:15,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:15,315 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,37823,1683694419534': 2023-05-10 04:55:15,315 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37823,1683694419534' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-05-10 04:55:15,316 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-05-10 04:55:15,316 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-10 04:55:15,316 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-10 04:55:15,316 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-05-10 04:55:15,316 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-10 04:55:15,317 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-10 04:55:15,317 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-10 04:55:15,317 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-10 04:55:15,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-10 04:55:15,317 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:15,317 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-10 04:55:15,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:55:15,317 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-10 04:55:15,317 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:15,317 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-10 04:55:15,317 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-10 04:55:15,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-10 04:55:15,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-10 04:55:15,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:55:15,317 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:15,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-10 04:55:15,318 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:15,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:15,318 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:15,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:15,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-10 04:55:15,318 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:15,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:55:15,318 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:15,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:15,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-10 04:55:15,318 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:15,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-10 04:55:15,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-10 04:55:15,319 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:15,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:55:15,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-10 04:55:15,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:15,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:15,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:15,320 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:15,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:55:15,321 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:15,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-10 04:55:15,321 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:15,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:15,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:15,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:15,327 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:15,328 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:15,328 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:15,328 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:15,328 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:15,328 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:15,328 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:15,328 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-05-10 04:55:15,328 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-10 04:55:15,328 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:55:15,328 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:15,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:15,328 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-05-10 04:55:15,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:15,328 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:15,328 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-10 04:55:15,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:15,328 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-10 04:55:15,328 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:15,328 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:15,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:15,328 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:15,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:15,328 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:15,329 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:15,329 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-10 04:55:15,329 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:15,329 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-10 04:55:15,329 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:15,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:15,329 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:15,329 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:15,329 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-10 04:55:15,329 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-10 04:55:15,429 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-10 04:55:15,429 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-10 04:55:15,441 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-10 04:55:15,442 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-05-10 04:55:15,442 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-05-10 04:55:15,442 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-05-10 04:55:15,442 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:55:15,442 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-05-10 04:55:15,442 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-05-10 04:55:15,443 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-10 04:55:15,443 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-10 04:55:15,444 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:15,444 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:15,444 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:15,444 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:15,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:15,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:15,444 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:15,444 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:15,444 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:15,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:15,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-10 04:55:15,445 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:15,445 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-10 04:55:15,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-10 04:55:15,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-10 04:55:15,445 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-10 04:55:15,445 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-10 04:55:15,445 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-10 04:55:15,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:55:15,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-10 04:55:15,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:55:15,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:55:15,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-10 04:55:15,445 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-05-10 04:55:15,446 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-05-10 04:55:15,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-10 04:55:15,446 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-05-10 04:55:15,446 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-05-10 04:55:15,446 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-05-10 04:55:15,446 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-05-10 04:55:15,446 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-05-10 04:55:15,446 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-05-10 04:55:15,446 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-05-10 04:55:15,446 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-05-10 04:55:15,446 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:55:15,446 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:55:15,446 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-05-10 04:55:15,446 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-05-10 04:55:15,446 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-05-10 04:55:15,446 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-05-10 04:55:15,446 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-05-10 04:55:15,446 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-10 04:55:15,446 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-10 04:55:15,446 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37995,1683694419499' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-05-10 04:55:15,446 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1683694514393.28e42ae67401b1d1f2840cbe891ccf89. 2023-05-10 04:55:15,446 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:55:15,446 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1683694514393.28e42ae67401b1d1f2840cbe891ccf89. started... 2023-05-10 04:55:15,446 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-05-10 04:55:15,446 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-05-10 04:55:15,447 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36665,1683694419452' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-05-10 04:55:15,447 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 28e42ae67401b1d1f2840cbe891ccf89 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-10 04:55:15,448 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-10 04:55:15,449 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-10 04:55:15,449 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-10 04:55:15,449 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:55:15,449 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-10 04:55:15,449 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:55:15,455 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testInvalidSplitKey/28e42ae67401b1d1f2840cbe891ccf89/.tmp/cf1/63487b6e423a4aa2a557067da66faf57 2023-05-10 04:55:15,473 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testInvalidSplitKey/28e42ae67401b1d1f2840cbe891ccf89/.tmp/cf2/ad19e9158e8c477eb4fbe865c68e2a5f 2023-05-10 04:55:15,478 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testInvalidSplitKey/28e42ae67401b1d1f2840cbe891ccf89/.tmp/cf1/63487b6e423a4aa2a557067da66faf57 as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testInvalidSplitKey/28e42ae67401b1d1f2840cbe891ccf89/cf1/63487b6e423a4aa2a557067da66faf57 2023-05-10 04:55:15,481 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testInvalidSplitKey/28e42ae67401b1d1f2840cbe891ccf89/cf1/63487b6e423a4aa2a557067da66faf57, entries=10, sequenceid=45, filesize=5.0 K 2023-05-10 04:55:15,482 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testInvalidSplitKey/28e42ae67401b1d1f2840cbe891ccf89/.tmp/cf2/ad19e9158e8c477eb4fbe865c68e2a5f as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testInvalidSplitKey/28e42ae67401b1d1f2840cbe891ccf89/cf2/ad19e9158e8c477eb4fbe865c68e2a5f 2023-05-10 04:55:15,485 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testInvalidSplitKey/28e42ae67401b1d1f2840cbe891ccf89/cf2/ad19e9158e8c477eb4fbe865c68e2a5f, entries=10, sequenceid=45, filesize=5.0 K 2023-05-10 04:55:15,486 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 28e42ae67401b1d1f2840cbe891ccf89 in 39ms, sequenceid=45, compaction requested=true 2023-05-10 04:55:15,486 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 28e42ae67401b1d1f2840cbe891ccf89: 2023-05-10 04:55:15,486 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1683694514393.28e42ae67401b1d1f2840cbe891ccf89. 2023-05-10 04:55:15,486 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-10 04:55:15,486 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-10 04:55:15,486 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:15,486 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-05-10 04:55:15,486 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37823,1683694419534' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-05-10 04:55:15,489 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:15,489 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-10 04:55:15,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:15,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:55:15,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:55:15,489 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-10 04:55:15,489 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:55:15,489 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:55:15,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:55:15,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-10 04:55:15,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:15,490 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:15,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:15,491 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:55:15,491 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37823,1683694419534' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-05-10 04:55:15,491 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@78dd12fa[Count = 0] remaining members to acquire global barrier 2023-05-10 04:55:15,491 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-05-10 04:55:15,491 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-10 04:55:15,492 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-10 04:55:15,492 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-10 04:55:15,492 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-10 04:55:15,492 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-10 04:55:15,492 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-10 04:55:15,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-10 04:55:15,492 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-10 04:55:15,492 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-05-10 04:55:15,492 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:15,493 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-10 04:55:15,492 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-10 04:55:15,493 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-05-10 04:55:15,493 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-05-10 04:55:15,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-10 04:55:15,493 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-05-10 04:55:15,493 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,36665,1683694419452' in zk 2023-05-10 04:55:15,493 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,37995,1683694419499' in zk 2023-05-10 04:55:15,493 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-05-10 04:55:15,493 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-05-10 04:55:15,493 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,37823,1683694419534' in zk 2023-05-10 04:55:15,494 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-05-10 04:55:15,494 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-05-10 04:55:15,494 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:55:15,494 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:55:15,494 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-05-10 04:55:15,494 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-05-10 04:55:15,495 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:15,495 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-05-10 04:55:15,495 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:55:15,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:15,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:55:15,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:55:15,495 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-05-10 04:55:15,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:55:15,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:55:15,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-10 04:55:15,496 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:15,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:15,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:15,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:55:15,497 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-10 04:55:15,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:15,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:15,498 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:15,499 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,37823,1683694419534': 2023-05-10 04:55:15,499 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37823,1683694419534' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-05-10 04:55:15,499 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-05-10 04:55:15,499 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-10 04:55:15,499 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-10 04:55:15,499 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-05-10 04:55:15,499 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-10 04:55:15,502 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-10 04:55:15,502 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-10 04:55:15,502 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-10 04:55:15,502 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:15,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-10 04:55:15,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:55:15,502 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-10 04:55:15,502 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:15,502 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-10 04:55:15,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-10 04:55:15,502 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-10 04:55:15,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-10 04:55:15,503 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-10 04:55:15,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-10 04:55:15,502 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:55:15,503 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:15,503 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:15,503 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:15,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:15,503 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:15,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:15,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:15,503 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:15,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-10 04:55:15,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:55:15,503 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-10 04:55:15,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-10 04:55:15,504 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:15,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-10 04:55:15,504 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:15,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:55:15,504 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-10 04:55:15,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:15,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:15,505 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:15,506 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:15,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:55:15,506 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:15,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-10 04:55:15,506 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:15,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:15,506 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:15,507 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:15,509 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:15,509 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:15,509 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:15,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:15,509 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:15,509 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:15,509 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:15,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:15,509 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:15,509 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:15,509 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:15,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:15,509 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:15,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:15,509 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:15,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:15,510 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:15,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:15,511 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:15,511 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-10 04:55:15,511 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:15,511 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:15,511 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:55:15,511 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-10 04:55:15,511 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-05-10 04:55:15,511 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-10 04:55:15,511 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:15,511 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-10 04:55:15,512 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:15,512 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:15,512 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-10 04:55:15,512 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-10 04:55:15,512 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-05-10 04:55:15,512 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-10 04:55:15,612 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-10 04:55:15,613 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-10 04:55:15,624 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-10 04:55:15,626 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-05-10 04:55:15,626 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-05-10 04:55:15,626 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-05-10 04:55:15,626 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:55:15,626 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-05-10 04:55:15,626 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-05-10 04:55:15,626 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-10 04:55:15,627 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-10 04:55:15,628 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:15,628 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:15,628 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:15,628 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:15,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:15,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:15,628 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:15,628 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:15,628 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:15,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:15,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-10 04:55:15,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-10 04:55:15,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-10 04:55:15,628 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:15,628 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-10 04:55:15,629 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-10 04:55:15,629 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-10 04:55:15,629 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-10 04:55:15,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:55:15,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-10 04:55:15,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:55:15,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-10 04:55:15,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:55:15,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-10 04:55:15,629 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-05-10 04:55:15,629 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-05-10 04:55:15,629 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-05-10 04:55:15,629 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-05-10 04:55:15,629 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-05-10 04:55:15,629 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-05-10 04:55:15,629 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-05-10 04:55:15,630 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-05-10 04:55:15,630 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-05-10 04:55:15,630 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-05-10 04:55:15,630 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:55:15,630 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-05-10 04:55:15,630 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-05-10 04:55:15,630 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-05-10 04:55:15,630 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-10 04:55:15,630 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1683694514393.28e42ae67401b1d1f2840cbe891ccf89. 2023-05-10 04:55:15,630 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:55:15,630 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1683694514393.28e42ae67401b1d1f2840cbe891ccf89. started... 2023-05-10 04:55:15,630 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-10 04:55:15,630 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:55:15,630 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-05-10 04:55:15,630 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 28e42ae67401b1d1f2840cbe891ccf89 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-10 04:55:15,630 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-05-10 04:55:15,630 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-05-10 04:55:15,630 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-05-10 04:55:15,630 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37995,1683694419499' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-05-10 04:55:15,630 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36665,1683694419452' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-05-10 04:55:15,632 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-10 04:55:15,634 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-10 04:55:15,634 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-10 04:55:15,634 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:55:15,634 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-10 04:55:15,634 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:55:15,641 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testInvalidSplitKey/28e42ae67401b1d1f2840cbe891ccf89/.tmp/cf1/d7be1202eaec4fffad9530cee703a9b3 2023-05-10 04:55:15,651 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testInvalidSplitKey/28e42ae67401b1d1f2840cbe891ccf89/.tmp/cf2/3212ed0ecf284765ac14fd59e189cf72 2023-05-10 04:55:15,655 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testInvalidSplitKey/28e42ae67401b1d1f2840cbe891ccf89/.tmp/cf1/d7be1202eaec4fffad9530cee703a9b3 as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testInvalidSplitKey/28e42ae67401b1d1f2840cbe891ccf89/cf1/d7be1202eaec4fffad9530cee703a9b3 2023-05-10 04:55:15,659 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testInvalidSplitKey/28e42ae67401b1d1f2840cbe891ccf89/cf1/d7be1202eaec4fffad9530cee703a9b3, entries=10, sequenceid=58, filesize=5.0 K 2023-05-10 04:55:15,659 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testInvalidSplitKey/28e42ae67401b1d1f2840cbe891ccf89/.tmp/cf2/3212ed0ecf284765ac14fd59e189cf72 as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testInvalidSplitKey/28e42ae67401b1d1f2840cbe891ccf89/cf2/3212ed0ecf284765ac14fd59e189cf72 2023-05-10 04:55:15,663 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testInvalidSplitKey/28e42ae67401b1d1f2840cbe891ccf89/cf2/3212ed0ecf284765ac14fd59e189cf72, entries=10, sequenceid=58, filesize=5.0 K 2023-05-10 04:55:15,663 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 28e42ae67401b1d1f2840cbe891ccf89 in 33ms, sequenceid=58, compaction requested=true 2023-05-10 04:55:15,664 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 28e42ae67401b1d1f2840cbe891ccf89: 2023-05-10 04:55:15,664 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1683694514393.28e42ae67401b1d1f2840cbe891ccf89. 2023-05-10 04:55:15,664 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-10 04:55:15,664 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-10 04:55:15,664 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:15,664 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-05-10 04:55:15,664 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37823,1683694419534' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-05-10 04:55:15,665 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:15,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:15,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:55:15,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:55:15,665 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-10 04:55:15,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:55:15,666 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-10 04:55:15,666 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:55:15,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:55:15,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-10 04:55:15,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:15,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:15,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:15,667 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:55:15,667 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37823,1683694419534' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-05-10 04:55:15,667 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3c319c74[Count = 0] remaining members to acquire global barrier 2023-05-10 04:55:15,667 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-05-10 04:55:15,667 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-10 04:55:15,668 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-10 04:55:15,668 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-10 04:55:15,668 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-10 04:55:15,668 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-10 04:55:15,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-10 04:55:15,668 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-10 04:55:15,669 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-05-10 04:55:15,669 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:15,669 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-10 04:55:15,669 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-05-10 04:55:15,669 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-10 04:55:15,669 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,37823,1683694419534' in zk 2023-05-10 04:55:15,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-10 04:55:15,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-10 04:55:15,669 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-05-10 04:55:15,669 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-05-10 04:55:15,669 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-05-10 04:55:15,669 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,36665,1683694419452' in zk 2023-05-10 04:55:15,669 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-05-10 04:55:15,669 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,37995,1683694419499' in zk 2023-05-10 04:55:15,674 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:15,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:15,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:55:15,674 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-05-10 04:55:15,674 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:55:15,674 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:55:15,674 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-05-10 04:55:15,677 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-05-10 04:55:15,677 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:55:15,677 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-05-10 04:55:15,678 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-05-10 04:55:15,678 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:55:15,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:55:15,678 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-05-10 04:55:15,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:55:15,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-10 04:55:15,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:15,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:15,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:15,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:55:15,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-10 04:55:15,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:15,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:15,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:15,680 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,37823,1683694419534': 2023-05-10 04:55:15,680 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37823,1683694419534' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-05-10 04:55:15,680 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-05-10 04:55:15,680 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-10 04:55:15,680 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-10 04:55:15,680 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-05-10 04:55:15,680 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-10 04:55:15,683 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-10 04:55:15,683 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-10 04:55:15,683 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-10 04:55:15,683 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-10 04:55:15,683 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-10 04:55:15,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-10 04:55:15,683 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-10 04:55:15,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-10 04:55:15,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-10 04:55:15,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:55:15,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:55:15,683 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-10 04:55:15,683 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:15,683 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:15,683 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:15,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-10 04:55:15,683 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:15,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:55:15,683 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:15,683 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:15,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:15,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:15,684 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:15,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:15,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-10 04:55:15,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-10 04:55:15,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-10 04:55:15,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:55:15,684 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:15,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-10 04:55:15,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-10 04:55:15,684 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:15,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:15,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:15,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:15,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:55:15,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-10 04:55:15,686 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:15,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:15,686 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:15,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:15,686 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:15,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:15,688 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:15,688 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:15,688 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-10 04:55:15,689 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:15,688 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:15,689 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:15,689 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:15,689 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:15,689 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:15,689 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:15,689 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:15,689 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:55:15,689 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:15,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:15,689 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-05-10 04:55:15,689 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-10 04:55:15,689 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:15,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:15,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:15,689 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:15,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:15,689 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-10 04:55:15,689 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-05-10 04:55:15,690 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-10 04:55:15,689 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:15,690 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:15,690 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-10 04:55:15,690 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:15,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:15,690 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:15,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:15,690 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:15,690 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-10 04:55:15,690 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-10 04:55:15,790 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-10 04:55:15,791 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-10 04:55:15,801 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-10 04:55:15,802 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-05-10 04:55:15,802 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-05-10 04:55:15,803 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-05-10 04:55:15,803 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:55:15,803 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-05-10 04:55:15,803 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-05-10 04:55:15,803 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-10 04:55:15,803 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-10 04:55:15,804 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:15,804 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:15,804 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:15,804 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:15,804 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:15,804 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:15,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:15,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:15,804 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:15,805 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-10 04:55:15,804 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:15,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:15,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-10 04:55:15,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-10 04:55:15,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-10 04:55:15,805 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-10 04:55:15,805 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-10 04:55:15,805 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-10 04:55:15,805 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:55:15,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-10 04:55:15,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:55:15,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-10 04:55:15,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:55:15,806 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-10 04:55:15,806 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-05-10 04:55:15,806 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-05-10 04:55:15,806 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-05-10 04:55:15,806 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-05-10 04:55:15,806 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-05-10 04:55:15,806 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-05-10 04:55:15,806 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-05-10 04:55:15,806 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-05-10 04:55:15,806 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:55:15,806 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-05-10 04:55:15,806 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-05-10 04:55:15,806 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-05-10 04:55:15,806 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:55:15,806 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-05-10 04:55:15,806 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-10 04:55:15,807 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-10 04:55:15,806 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-05-10 04:55:15,807 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-05-10 04:55:15,807 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1683694514393.28e42ae67401b1d1f2840cbe891ccf89. 2023-05-10 04:55:15,806 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-05-10 04:55:15,807 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:55:15,807 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1683694514393.28e42ae67401b1d1f2840cbe891ccf89. started... 2023-05-10 04:55:15,807 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-05-10 04:55:15,807 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-05-10 04:55:15,807 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36665,1683694419452' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-05-10 04:55:15,807 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37995,1683694419499' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-05-10 04:55:15,807 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 28e42ae67401b1d1f2840cbe891ccf89 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-10 04:55:15,808 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-10 04:55:15,808 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-10 04:55:15,809 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-10 04:55:15,809 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:55:15,809 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-10 04:55:15,809 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:55:15,815 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testInvalidSplitKey/28e42ae67401b1d1f2840cbe891ccf89/.tmp/cf1/bd02bc42db8f4378a3fcd4cf31ba943e 2023-05-10 04:55:15,827 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testInvalidSplitKey/28e42ae67401b1d1f2840cbe891ccf89/.tmp/cf2/ebde51577b17453a913b07c59326f52d 2023-05-10 04:55:15,831 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testInvalidSplitKey/28e42ae67401b1d1f2840cbe891ccf89/.tmp/cf1/bd02bc42db8f4378a3fcd4cf31ba943e as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testInvalidSplitKey/28e42ae67401b1d1f2840cbe891ccf89/cf1/bd02bc42db8f4378a3fcd4cf31ba943e 2023-05-10 04:55:15,835 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testInvalidSplitKey/28e42ae67401b1d1f2840cbe891ccf89/cf1/bd02bc42db8f4378a3fcd4cf31ba943e, entries=10, sequenceid=71, filesize=5.0 K 2023-05-10 04:55:15,836 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testInvalidSplitKey/28e42ae67401b1d1f2840cbe891ccf89/.tmp/cf2/ebde51577b17453a913b07c59326f52d as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testInvalidSplitKey/28e42ae67401b1d1f2840cbe891ccf89/cf2/ebde51577b17453a913b07c59326f52d 2023-05-10 04:55:15,839 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testInvalidSplitKey/28e42ae67401b1d1f2840cbe891ccf89/cf2/ebde51577b17453a913b07c59326f52d, entries=10, sequenceid=71, filesize=5.0 K 2023-05-10 04:55:15,840 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 28e42ae67401b1d1f2840cbe891ccf89 in 33ms, sequenceid=71, compaction requested=true 2023-05-10 04:55:15,840 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 28e42ae67401b1d1f2840cbe891ccf89: 2023-05-10 04:55:15,840 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1683694514393.28e42ae67401b1d1f2840cbe891ccf89. 2023-05-10 04:55:15,840 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-10 04:55:15,840 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-10 04:55:15,840 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:15,840 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-05-10 04:55:15,840 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37823,1683694419534' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-05-10 04:55:15,842 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:15,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:15,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:55:15,842 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-10 04:55:15,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:55:15,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:55:15,843 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-10 04:55:15,843 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:55:15,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:55:15,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-10 04:55:15,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:15,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:15,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:15,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:55:15,844 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37823,1683694419534' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-05-10 04:55:15,844 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6546cbd8[Count = 0] remaining members to acquire global barrier 2023-05-10 04:55:15,844 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-05-10 04:55:15,844 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-10 04:55:15,845 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-10 04:55:15,845 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-10 04:55:15,845 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-10 04:55:15,845 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-10 04:55:15,845 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-10 04:55:15,845 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:15,845 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-10 04:55:15,845 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-10 04:55:15,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-10 04:55:15,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-10 04:55:15,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-10 04:55:15,846 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-05-10 04:55:15,846 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-05-10 04:55:15,846 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-05-10 04:55:15,846 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-05-10 04:55:15,846 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-05-10 04:55:15,846 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,36665,1683694419452' in zk 2023-05-10 04:55:15,846 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,37995,1683694419499' in zk 2023-05-10 04:55:15,846 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-05-10 04:55:15,846 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase4.apache.org,37823,1683694419534' in zk 2023-05-10 04:55:15,847 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-05-10 04:55:15,847 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:55:15,847 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-05-10 04:55:15,848 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-05-10 04:55:15,848 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:55:15,848 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-05-10 04:55:15,848 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:15,848 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:55:15,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:15,848 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-05-10 04:55:15,848 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-05-10 04:55:15,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:55:15,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:55:15,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:55:15,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:55:15,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-10 04:55:15,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:15,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:15,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:15,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:55:15,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-10 04:55:15,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:15,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:15,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:15,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase4.apache.org,37823,1683694419534': 2023-05-10 04:55:15,851 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37823,1683694419534' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-05-10 04:55:15,851 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-05-10 04:55:15,851 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-10 04:55:15,851 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-10 04:55:15,851 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-05-10 04:55:15,851 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-10 04:55:15,854 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-10 04:55:15,854 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-10 04:55:15,854 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-10 04:55:15,854 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-10 04:55:15,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-10 04:55:15,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:55:15,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:55:15,854 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:15,854 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-10 04:55:15,854 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:15,854 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-10 04:55:15,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-10 04:55:15,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-10 04:55:15,854 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:15,854 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-10 04:55:15,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:55:15,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-10 04:55:15,855 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:15,855 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:15,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:15,855 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:15,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-10 04:55:15,855 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:15,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:15,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:15,855 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:15,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-10 04:55:15,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-10 04:55:15,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-10 04:55:15,855 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:15,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:55:15,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-10 04:55:15,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:15,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:15,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:15,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:55:15,857 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:15,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-05-10 04:55:15,857 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:15,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:15,858 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:15,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:15,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:15,860 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:15,860 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-10 04:55:15,860 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:15,860 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:15,860 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:15,860 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:15,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:15,860 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:55:15,860 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:15,860 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:15,860 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:15,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:15,860 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:15,860 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:15,860 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:15,860 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:15,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:15,860 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:15,860 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-05-10 04:55:15,861 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-10 04:55:15,861 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:15,861 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:15,861 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-05-10 04:55:15,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:15,861 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:15,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:15,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:15,861 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-10 04:55:15,861 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-05-10 04:55:15,861 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-10 04:55:15,861 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:15,861 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:15,861 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-05-10 04:55:15,861 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-05-10 04:55:15,961 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-10 04:55:15,962 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-10 04:55:15,968 INFO [Listener at localhost/45007] assignment.SplitTableRegionProcedure(206): testInvalidSplitKey,,1683694514393.28e42ae67401b1d1f2840cbe891ccf89. splitKey isn't explicitly specified, will try to find a best split key from RS jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:15,970 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37823] regionserver.StoreUtils(129): cannot split hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testInvalidSplitKey/28e42ae67401b1d1f2840cbe891ccf89/cf1/42f3469d25b14586ad2d722e4281417a because midkey is the same as first or last row 2023-05-10 04:55:15,970 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37823] regionserver.StoreUtils(129): cannot split hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testInvalidSplitKey/28e42ae67401b1d1f2840cbe891ccf89/cf2/1df608e0fef74a0b994e80ed5c6bf96c because midkey is the same as first or last row 2023-05-10 04:55:15,970 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37823] regionserver.HRegion(2745): Flushing 28e42ae67401b1d1f2840cbe891ccf89 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-05-10 04:55:15,978 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37823] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testInvalidSplitKey/28e42ae67401b1d1f2840cbe891ccf89/.tmp/cf1/dbb87477d1214aeca0b9df7698b70e32 2023-05-10 04:55:15,989 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37823] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testInvalidSplitKey/28e42ae67401b1d1f2840cbe891ccf89/.tmp/cf2/4975294f7ff2428198a377d68b2b8be2 2023-05-10 04:55:15,992 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37823] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testInvalidSplitKey/28e42ae67401b1d1f2840cbe891ccf89/.tmp/cf1/dbb87477d1214aeca0b9df7698b70e32 as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testInvalidSplitKey/28e42ae67401b1d1f2840cbe891ccf89/cf1/dbb87477d1214aeca0b9df7698b70e32 2023-05-10 04:55:15,996 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37823] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testInvalidSplitKey/28e42ae67401b1d1f2840cbe891ccf89/cf1/dbb87477d1214aeca0b9df7698b70e32, entries=8, sequenceid=82, filesize=5.0 K 2023-05-10 04:55:15,996 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37823] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testInvalidSplitKey/28e42ae67401b1d1f2840cbe891ccf89/.tmp/cf2/4975294f7ff2428198a377d68b2b8be2 as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testInvalidSplitKey/28e42ae67401b1d1f2840cbe891ccf89/cf2/4975294f7ff2428198a377d68b2b8be2 2023-05-10 04:55:16,000 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37823] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testInvalidSplitKey/28e42ae67401b1d1f2840cbe891ccf89/cf2/4975294f7ff2428198a377d68b2b8be2, entries=8, sequenceid=82, filesize=5.0 K 2023-05-10 04:55:16,001 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37823] regionserver.HRegion(2948): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 28e42ae67401b1d1f2840cbe891ccf89 in 30ms, sequenceid=82, compaction requested=true 2023-05-10 04:55:16,001 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37823] regionserver.HRegion(2446): Flush status journal for 28e42ae67401b1d1f2840cbe891ccf89: 2023-05-10 04:55:16,002 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37823] regionserver.StoreUtils(129): cannot split hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testInvalidSplitKey/28e42ae67401b1d1f2840cbe891ccf89/cf1/42f3469d25b14586ad2d722e4281417a because midkey is the same as first or last row 2023-05-10 04:55:16,002 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=37823] regionserver.StoreUtils(129): cannot split hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testInvalidSplitKey/28e42ae67401b1d1f2840cbe891ccf89/cf2/1df608e0fef74a0b994e80ed5c6bf96c because midkey is the same as first or last row 2023-05-10 04:55:16,002 DEBUG [Listener at localhost/45007] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:16,002 DEBUG [Listener at localhost/45007] assignment.TestSplitTableRegionProcedure(438): Expected Split procedure construction failure: Region not splittable because bestSplitPoint = null, maybe table is too small for auto split. For force split, try specifying split row 2023-05-10 04:55:16,002 WARN [Listener at localhost/45007] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-05-10 04:55:16,004 INFO [Listener at localhost/45007] client.HBaseAdmin$15(890): Started disable of testInvalidSplitKey 2023-05-10 04:55:16,004 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testInvalidSplitKey 2023-05-10 04:55:16,005 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] procedure2.ProcedureExecutor(1029): Stored pid=105, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testInvalidSplitKey 2023-05-10 04:55:16,007 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-05-10 04:55:16,007 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683694516007"}]},"ts":"1683694516007"} 2023-05-10 04:55:16,008 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=DISABLING in hbase:meta 2023-05-10 04:55:16,010 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testInvalidSplitKey to state=DISABLING 2023-05-10 04:55:16,010 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testInvalidSplitKey, region=28e42ae67401b1d1f2840cbe891ccf89, UNASSIGN}] 2023-05-10 04:55:16,011 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testInvalidSplitKey, region=28e42ae67401b1d1f2840cbe891ccf89, UNASSIGN 2023-05-10 04:55:16,012 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=28e42ae67401b1d1f2840cbe891ccf89, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:16,012 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testInvalidSplitKey,,1683694514393.28e42ae67401b1d1f2840cbe891ccf89.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1683694516012"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683694516012"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683694516012"}]},"ts":"1683694516012"} 2023-05-10 04:55:16,013 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=107, ppid=106, state=RUNNABLE; CloseRegionProcedure 28e42ae67401b1d1f2840cbe891ccf89, server=jenkins-hbase4.apache.org,37823,1683694419534}] 2023-05-10 04:55:16,108 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-05-10 04:55:16,165 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 28e42ae67401b1d1f2840cbe891ccf89 2023-05-10 04:55:16,165 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 28e42ae67401b1d1f2840cbe891ccf89, disabling compactions & flushes 2023-05-10 04:55:16,166 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testInvalidSplitKey,,1683694514393.28e42ae67401b1d1f2840cbe891ccf89. 2023-05-10 04:55:16,166 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testInvalidSplitKey,,1683694514393.28e42ae67401b1d1f2840cbe891ccf89. 2023-05-10 04:55:16,166 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testInvalidSplitKey,,1683694514393.28e42ae67401b1d1f2840cbe891ccf89. after waiting 0 ms 2023-05-10 04:55:16,166 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testInvalidSplitKey,,1683694514393.28e42ae67401b1d1f2840cbe891ccf89. 2023-05-10 04:55:16,169 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testInvalidSplitKey/28e42ae67401b1d1f2840cbe891ccf89/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-05-10 04:55:16,170 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-10 04:55:16,170 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testInvalidSplitKey,,1683694514393.28e42ae67401b1d1f2840cbe891ccf89. 2023-05-10 04:55:16,170 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 28e42ae67401b1d1f2840cbe891ccf89: 2023-05-10 04:55:16,171 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 28e42ae67401b1d1f2840cbe891ccf89 2023-05-10 04:55:16,172 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=28e42ae67401b1d1f2840cbe891ccf89, regionState=CLOSED 2023-05-10 04:55:16,172 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testInvalidSplitKey,,1683694514393.28e42ae67401b1d1f2840cbe891ccf89.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1683694516172"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683694516172"}]},"ts":"1683694516172"} 2023-05-10 04:55:16,174 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=107, resume processing ppid=106 2023-05-10 04:55:16,174 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=107, ppid=106, state=SUCCESS; CloseRegionProcedure 28e42ae67401b1d1f2840cbe891ccf89, server=jenkins-hbase4.apache.org,37823,1683694419534 in 160 msec 2023-05-10 04:55:16,175 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=106, resume processing ppid=105 2023-05-10 04:55:16,175 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=106, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=testInvalidSplitKey, region=28e42ae67401b1d1f2840cbe891ccf89, UNASSIGN in 164 msec 2023-05-10 04:55:16,176 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683694516176"}]},"ts":"1683694516176"} 2023-05-10 04:55:16,177 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=DISABLED in hbase:meta 2023-05-10 04:55:16,179 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testInvalidSplitKey to state=DISABLED 2023-05-10 04:55:16,180 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey in 175 msec 2023-05-10 04:55:16,309 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(1227): Checking to see if procedure is done pid=105 2023-05-10 04:55:16,310 INFO [Listener at localhost/45007] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testInvalidSplitKey, procId: 105 completed 2023-05-10 04:55:16,310 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete testInvalidSplitKey 2023-05-10 04:55:16,311 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] procedure2.ProcedureExecutor(1029): Stored pid=108, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testInvalidSplitKey 2023-05-10 04:55:16,312 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=108, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-05-10 04:55:16,313 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=108, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-05-10 04:55:16,313 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(1227): Checking to see if procedure is done pid=108 2023-05-10 04:55:16,316 DEBUG [HFileArchiver-11] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testInvalidSplitKey/28e42ae67401b1d1f2840cbe891ccf89 2023-05-10 04:55:16,317 DEBUG [HFileArchiver-11] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testInvalidSplitKey/28e42ae67401b1d1f2840cbe891ccf89/cf1, FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testInvalidSplitKey/28e42ae67401b1d1f2840cbe891ccf89/cf2, FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testInvalidSplitKey/28e42ae67401b1d1f2840cbe891ccf89/recovered.edits] 2023-05-10 04:55:16,321 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testInvalidSplitKey/28e42ae67401b1d1f2840cbe891ccf89/cf1/42f3469d25b14586ad2d722e4281417a to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testInvalidSplitKey/28e42ae67401b1d1f2840cbe891ccf89/cf1/42f3469d25b14586ad2d722e4281417a 2023-05-10 04:55:16,322 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testInvalidSplitKey/28e42ae67401b1d1f2840cbe891ccf89/cf1/63487b6e423a4aa2a557067da66faf57 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testInvalidSplitKey/28e42ae67401b1d1f2840cbe891ccf89/cf1/63487b6e423a4aa2a557067da66faf57 2023-05-10 04:55:16,323 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testInvalidSplitKey/28e42ae67401b1d1f2840cbe891ccf89/cf1/8c06e6af40d343b890bafa9b20b78e1e to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testInvalidSplitKey/28e42ae67401b1d1f2840cbe891ccf89/cf1/8c06e6af40d343b890bafa9b20b78e1e 2023-05-10 04:55:16,324 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testInvalidSplitKey/28e42ae67401b1d1f2840cbe891ccf89/cf1/bd02bc42db8f4378a3fcd4cf31ba943e to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testInvalidSplitKey/28e42ae67401b1d1f2840cbe891ccf89/cf1/bd02bc42db8f4378a3fcd4cf31ba943e 2023-05-10 04:55:16,326 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testInvalidSplitKey/28e42ae67401b1d1f2840cbe891ccf89/cf1/d7be1202eaec4fffad9530cee703a9b3 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testInvalidSplitKey/28e42ae67401b1d1f2840cbe891ccf89/cf1/d7be1202eaec4fffad9530cee703a9b3 2023-05-10 04:55:16,327 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testInvalidSplitKey/28e42ae67401b1d1f2840cbe891ccf89/cf1/dbb87477d1214aeca0b9df7698b70e32 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testInvalidSplitKey/28e42ae67401b1d1f2840cbe891ccf89/cf1/dbb87477d1214aeca0b9df7698b70e32 2023-05-10 04:55:16,328 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testInvalidSplitKey/28e42ae67401b1d1f2840cbe891ccf89/cf1/ec297150d49e422baece5d4186fed525 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testInvalidSplitKey/28e42ae67401b1d1f2840cbe891ccf89/cf1/ec297150d49e422baece5d4186fed525 2023-05-10 04:55:16,330 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testInvalidSplitKey/28e42ae67401b1d1f2840cbe891ccf89/cf2/1df608e0fef74a0b994e80ed5c6bf96c to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testInvalidSplitKey/28e42ae67401b1d1f2840cbe891ccf89/cf2/1df608e0fef74a0b994e80ed5c6bf96c 2023-05-10 04:55:16,331 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testInvalidSplitKey/28e42ae67401b1d1f2840cbe891ccf89/cf2/3212ed0ecf284765ac14fd59e189cf72 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testInvalidSplitKey/28e42ae67401b1d1f2840cbe891ccf89/cf2/3212ed0ecf284765ac14fd59e189cf72 2023-05-10 04:55:16,332 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testInvalidSplitKey/28e42ae67401b1d1f2840cbe891ccf89/cf2/4975294f7ff2428198a377d68b2b8be2 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testInvalidSplitKey/28e42ae67401b1d1f2840cbe891ccf89/cf2/4975294f7ff2428198a377d68b2b8be2 2023-05-10 04:55:16,333 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testInvalidSplitKey/28e42ae67401b1d1f2840cbe891ccf89/cf2/5ff7d2b62aa0473a8aa0ceeda965c0c0 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testInvalidSplitKey/28e42ae67401b1d1f2840cbe891ccf89/cf2/5ff7d2b62aa0473a8aa0ceeda965c0c0 2023-05-10 04:55:16,334 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testInvalidSplitKey/28e42ae67401b1d1f2840cbe891ccf89/cf2/667e01382a094633be211ae11ba65d04 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testInvalidSplitKey/28e42ae67401b1d1f2840cbe891ccf89/cf2/667e01382a094633be211ae11ba65d04 2023-05-10 04:55:16,335 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testInvalidSplitKey/28e42ae67401b1d1f2840cbe891ccf89/cf2/ad19e9158e8c477eb4fbe865c68e2a5f to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testInvalidSplitKey/28e42ae67401b1d1f2840cbe891ccf89/cf2/ad19e9158e8c477eb4fbe865c68e2a5f 2023-05-10 04:55:16,336 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testInvalidSplitKey/28e42ae67401b1d1f2840cbe891ccf89/cf2/ebde51577b17453a913b07c59326f52d to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testInvalidSplitKey/28e42ae67401b1d1f2840cbe891ccf89/cf2/ebde51577b17453a913b07c59326f52d 2023-05-10 04:55:16,339 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testInvalidSplitKey/28e42ae67401b1d1f2840cbe891ccf89/recovered.edits/85.seqid to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testInvalidSplitKey/28e42ae67401b1d1f2840cbe891ccf89/recovered.edits/85.seqid 2023-05-10 04:55:16,339 DEBUG [HFileArchiver-11] backup.HFileArchiver(596): Deleted hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testInvalidSplitKey/28e42ae67401b1d1f2840cbe891ccf89 2023-05-10 04:55:16,339 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testInvalidSplitKey regions 2023-05-10 04:55:16,341 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=108, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-05-10 04:55:16,342 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testInvalidSplitKey from hbase:meta 2023-05-10 04:55:16,346 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testInvalidSplitKey' descriptor. 2023-05-10 04:55:16,347 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=108, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-05-10 04:55:16,347 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testInvalidSplitKey' from region states. 2023-05-10 04:55:16,347 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testInvalidSplitKey,,1683694514393.28e42ae67401b1d1f2840cbe891ccf89.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1683694516347"}]},"ts":"9223372036854775807"} 2023-05-10 04:55:16,348 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 1 regions from META 2023-05-10 04:55:16,348 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 28e42ae67401b1d1f2840cbe891ccf89, NAME => 'testInvalidSplitKey,,1683694514393.28e42ae67401b1d1f2840cbe891ccf89.', STARTKEY => '', ENDKEY => ''}] 2023-05-10 04:55:16,348 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testInvalidSplitKey' as deleted. 2023-05-10 04:55:16,348 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1683694516348"}]},"ts":"9223372036854775807"} 2023-05-10 04:55:16,350 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testInvalidSplitKey state from META 2023-05-10 04:55:16,351 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=108, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-05-10 04:55:16,352 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey in 41 msec 2023-05-10 04:55:16,414 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(1227): Checking to see if procedure is done pid=108 2023-05-10 04:55:16,415 INFO [Listener at localhost/45007] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testInvalidSplitKey, procId: 108 completed 2023-05-10 04:55:16,428 INFO [Listener at localhost/45007] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testInvalidSplitKey Thread=368 (was 348) - Thread LEAK? -, OpenFileDescriptor=744 (was 658) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=177 (was 175) - SystemLoadAverage LEAK? -, ProcessCount=172 (was 172), AvailableMemoryMB=9552 (was 9556) 2023-05-10 04:55:16,440 INFO [Listener at localhost/45007] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testRecoveryAndDoubleExecution Thread=368, OpenFileDescriptor=744, MaxFileDescriptor=60000, SystemLoadAverage=177, ProcessCount=172, AvailableMemoryMB=9552 2023-05-10 04:55:16,441 WARN [Listener at localhost/45007] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-05-10 04:55:16,443 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-05-10 04:55:16,443 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-05-10 04:55:16,444 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(1029): Stored pid=109, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-05-10 04:55:16,445 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_PRE_OPERATION 2023-05-10 04:55:16,446 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-05-10 04:55:16,447 DEBUG [HFileArchiver-12] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:16,448 DEBUG [HFileArchiver-12] backup.HFileArchiver(153): Directory hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536 empty. 2023-05-10 04:55:16,448 DEBUG [HFileArchiver-12] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:16,448 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRecoveryAndDoubleExecution regions 2023-05-10 04:55:16,457 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testRecoveryAndDoubleExecution/.tabledesc/.tableinfo.0000000001 2023-05-10 04:55:16,458 INFO [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(7675): creating {ENCODED => ba55ae97fe6d26f2fd2a296837570536, NAME => 'testRecoveryAndDoubleExecution,,1683694516443.ba55ae97fe6d26f2fd2a296837570536.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testRecoveryAndDoubleExecution', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp 2023-05-10 04:55:16,464 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(866): Instantiated testRecoveryAndDoubleExecution,,1683694516443.ba55ae97fe6d26f2fd2a296837570536.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-10 04:55:16,464 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1604): Closing ba55ae97fe6d26f2fd2a296837570536, disabling compactions & flushes 2023-05-10 04:55:16,464 INFO [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1626): Closing region testRecoveryAndDoubleExecution,,1683694516443.ba55ae97fe6d26f2fd2a296837570536. 2023-05-10 04:55:16,464 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,,1683694516443.ba55ae97fe6d26f2fd2a296837570536. 2023-05-10 04:55:16,464 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1714): Acquired close lock on testRecoveryAndDoubleExecution,,1683694516443.ba55ae97fe6d26f2fd2a296837570536. after waiting 0 ms 2023-05-10 04:55:16,464 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1724): Updates disabled for region testRecoveryAndDoubleExecution,,1683694516443.ba55ae97fe6d26f2fd2a296837570536. 2023-05-10 04:55:16,464 INFO [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1838): Closed testRecoveryAndDoubleExecution,,1683694516443.ba55ae97fe6d26f2fd2a296837570536. 2023-05-10 04:55:16,464 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1558): Region close journal for ba55ae97fe6d26f2fd2a296837570536: 2023-05-10 04:55:16,466 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_ADD_TO_META 2023-05-10 04:55:16,467 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,,1683694516443.ba55ae97fe6d26f2fd2a296837570536.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1683694516467"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683694516467"}]},"ts":"1683694516467"} 2023-05-10 04:55:16,468 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-05-10 04:55:16,468 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-05-10 04:55:16,469 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683694516469"}]},"ts":"1683694516469"} 2023-05-10 04:55:16,469 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=ENABLING in hbase:meta 2023-05-10 04:55:16,475 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-05-10 04:55:16,476 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-05-10 04:55:16,476 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-05-10 04:55:16,476 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-05-10 04:55:16,476 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-05-10 04:55:16,476 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=110, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=ba55ae97fe6d26f2fd2a296837570536, ASSIGN}] 2023-05-10 04:55:16,477 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=110, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=ba55ae97fe6d26f2fd2a296837570536, ASSIGN 2023-05-10 04:55:16,478 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=110, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=ba55ae97fe6d26f2fd2a296837570536, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,36665,1683694419452; forceNewPlan=false, retain=false 2023-05-10 04:55:16,628 INFO [jenkins-hbase4:37903] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-05-10 04:55:16,629 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=110 updating hbase:meta row=ba55ae97fe6d26f2fd2a296837570536, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:16,629 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1683694516443.ba55ae97fe6d26f2fd2a296837570536.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1683694516629"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683694516629"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683694516629"}]},"ts":"1683694516629"} 2023-05-10 04:55:16,631 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=111, ppid=110, state=RUNNABLE; OpenRegionProcedure ba55ae97fe6d26f2fd2a296837570536, server=jenkins-hbase4.apache.org,36665,1683694419452}] 2023-05-10 04:55:16,787 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testRecoveryAndDoubleExecution,,1683694516443.ba55ae97fe6d26f2fd2a296837570536. 2023-05-10 04:55:16,787 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => ba55ae97fe6d26f2fd2a296837570536, NAME => 'testRecoveryAndDoubleExecution,,1683694516443.ba55ae97fe6d26f2fd2a296837570536.', STARTKEY => '', ENDKEY => ''} 2023-05-10 04:55:16,787 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-05-10 04:55:16,787 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRecoveryAndDoubleExecution ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:16,787 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testRecoveryAndDoubleExecution,,1683694516443.ba55ae97fe6d26f2fd2a296837570536.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-10 04:55:16,787 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:16,788 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:16,788 INFO [StoreOpener-ba55ae97fe6d26f2fd2a296837570536-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:16,790 DEBUG [StoreOpener-ba55ae97fe6d26f2fd2a296837570536-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf1 2023-05-10 04:55:16,790 DEBUG [StoreOpener-ba55ae97fe6d26f2fd2a296837570536-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf1 2023-05-10 04:55:16,790 INFO [StoreOpener-ba55ae97fe6d26f2fd2a296837570536-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ba55ae97fe6d26f2fd2a296837570536 columnFamilyName cf1 2023-05-10 04:55:16,790 INFO [StoreOpener-ba55ae97fe6d26f2fd2a296837570536-1] regionserver.HStore(310): Store=ba55ae97fe6d26f2fd2a296837570536/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-10 04:55:16,791 INFO [StoreOpener-ba55ae97fe6d26f2fd2a296837570536-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:16,791 DEBUG [StoreOpener-ba55ae97fe6d26f2fd2a296837570536-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf2 2023-05-10 04:55:16,791 DEBUG [StoreOpener-ba55ae97fe6d26f2fd2a296837570536-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf2 2023-05-10 04:55:16,792 INFO [StoreOpener-ba55ae97fe6d26f2fd2a296837570536-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ba55ae97fe6d26f2fd2a296837570536 columnFamilyName cf2 2023-05-10 04:55:16,792 INFO [StoreOpener-ba55ae97fe6d26f2fd2a296837570536-1] regionserver.HStore(310): Store=ba55ae97fe6d26f2fd2a296837570536/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-10 04:55:16,793 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:16,793 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:16,795 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRecoveryAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-05-10 04:55:16,796 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:16,797 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-05-10 04:55:16,797 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened ba55ae97fe6d26f2fd2a296837570536; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72860632, jitterRate=0.08570802211761475}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-05-10 04:55:16,798 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for ba55ae97fe6d26f2fd2a296837570536: 2023-05-10 04:55:16,798 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2334): Post open deploy tasks for testRecoveryAndDoubleExecution,,1683694516443.ba55ae97fe6d26f2fd2a296837570536., pid=111, masterSystemTime=1683694516782 2023-05-10 04:55:16,799 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2361): Finished post open deploy task for testRecoveryAndDoubleExecution,,1683694516443.ba55ae97fe6d26f2fd2a296837570536. 2023-05-10 04:55:16,800 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testRecoveryAndDoubleExecution,,1683694516443.ba55ae97fe6d26f2fd2a296837570536. 2023-05-10 04:55:16,800 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=110 updating hbase:meta row=ba55ae97fe6d26f2fd2a296837570536, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:16,800 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRecoveryAndDoubleExecution,,1683694516443.ba55ae97fe6d26f2fd2a296837570536.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1683694516800"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1683694516800"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1683694516800"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683694516800"}]},"ts":"1683694516800"} 2023-05-10 04:55:16,802 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=111, resume processing ppid=110 2023-05-10 04:55:16,803 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=111, ppid=110, state=SUCCESS; OpenRegionProcedure ba55ae97fe6d26f2fd2a296837570536, server=jenkins-hbase4.apache.org,36665,1683694419452 in 170 msec 2023-05-10 04:55:16,804 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=110, resume processing ppid=109 2023-05-10 04:55:16,804 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=110, ppid=109, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=ba55ae97fe6d26f2fd2a296837570536, ASSIGN in 327 msec 2023-05-10 04:55:16,805 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-05-10 04:55:16,805 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683694516805"}]},"ts":"1683694516805"} 2023-05-10 04:55:16,806 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=ENABLED in hbase:meta 2023-05-10 04:55:16,807 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_POST_OPERATION 2023-05-10 04:55:16,808 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution in 364 msec 2023-05-10 04:55:16,951 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-10 04:55:16,952 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-05-10 04:55:16,953 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-05-10 04:55:16,953 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:55:16,953 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-05-10 04:55:16,953 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-05-10 04:55:16,953 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-10 04:55:16,953 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-10 04:55:16,954 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:16,954 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:16,954 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:16,954 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:16,954 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:16,954 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:16,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:16,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:16,955 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:16,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:16,955 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:16,955 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-10 04:55:16,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-10 04:55:16,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-10 04:55:16,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-10 04:55:16,955 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-10 04:55:16,955 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-10 04:55:16,955 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-10 04:55:16,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:55:16,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-10 04:55:16,956 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-05-10 04:55:16,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:55:16,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-10 04:55:16,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:55:16,956 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-05-10 04:55:16,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-10 04:55:16,956 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-05-10 04:55:16,956 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:55:16,956 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-05-10 04:55:16,956 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-05-10 04:55:16,956 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-05-10 04:55:16,956 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-10 04:55:16,956 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-10 04:55:16,956 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-05-10 04:55:16,956 DEBUG [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1683694516443.ba55ae97fe6d26f2fd2a296837570536. 2023-05-10 04:55:16,956 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-05-10 04:55:16,956 DEBUG [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1683694516443.ba55ae97fe6d26f2fd2a296837570536. started... 2023-05-10 04:55:16,956 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-05-10 04:55:16,956 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-05-10 04:55:16,957 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:55:16,959 INFO [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing ba55ae97fe6d26f2fd2a296837570536 2/2 column families, dataSize=120 B heapSize=928 B 2023-05-10 04:55:16,956 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:55:16,959 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-05-10 04:55:16,959 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-05-10 04:55:16,959 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-05-10 04:55:16,959 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37995,1683694419499' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-05-10 04:55:16,959 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-05-10 04:55:16,959 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37823,1683694419534' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-05-10 04:55:16,960 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-10 04:55:16,960 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-10 04:55:16,961 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-10 04:55:16,961 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:55:16,961 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-10 04:55:16,961 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:55:16,969 INFO [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/.tmp/cf1/17ee8f61ba9d4f678887f8a52b767dbb 2023-05-10 04:55:16,974 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testRecoveryAndDoubleExecution' 2023-05-10 04:55:16,980 INFO [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/.tmp/cf2/eb76ae13368e4a23855b5680af181f32 2023-05-10 04:55:16,984 DEBUG [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/.tmp/cf1/17ee8f61ba9d4f678887f8a52b767dbb as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf1/17ee8f61ba9d4f678887f8a52b767dbb 2023-05-10 04:55:16,987 INFO [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf1/17ee8f61ba9d4f678887f8a52b767dbb, entries=2, sequenceid=6, filesize=4.8 K 2023-05-10 04:55:16,988 DEBUG [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/.tmp/cf2/eb76ae13368e4a23855b5680af181f32 as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf2/eb76ae13368e4a23855b5680af181f32 2023-05-10 04:55:16,991 INFO [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf2/eb76ae13368e4a23855b5680af181f32, entries=2, sequenceid=6, filesize=4.8 K 2023-05-10 04:55:16,991 INFO [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for ba55ae97fe6d26f2fd2a296837570536 in 32ms, sequenceid=6, compaction requested=false 2023-05-10 04:55:16,992 DEBUG [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for ba55ae97fe6d26f2fd2a296837570536: 2023-05-10 04:55:16,992 DEBUG [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1683694516443.ba55ae97fe6d26f2fd2a296837570536. 2023-05-10 04:55:16,992 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-10 04:55:16,992 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-10 04:55:16,992 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:16,992 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-05-10 04:55:16,992 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36665,1683694419452' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-05-10 04:55:16,997 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:16,997 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-10 04:55:16,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:16,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:55:16,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:55:16,997 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-10 04:55:16,997 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:55:16,997 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:55:16,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:55:16,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-10 04:55:16,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:16,998 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:16,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:16,999 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:55:16,999 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36665,1683694419452' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-05-10 04:55:16,999 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@76b848f0[Count = 0] remaining members to acquire global barrier 2023-05-10 04:55:16,999 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-05-10 04:55:16,999 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,000 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,000 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,000 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,000 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,000 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,000 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,001 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-05-10 04:55:17,001 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-05-10 04:55:17,001 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-05-10 04:55:17,000 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,001 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,001 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,37995,1683694419499' in zk 2023-05-10 04:55:17,001 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-05-10 04:55:17,001 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-05-10 04:55:17,001 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:17,001 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-10 04:55:17,001 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-05-10 04:55:17,001 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,37823,1683694419534' in zk 2023-05-10 04:55:17,001 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,36665,1683694419452' in zk 2023-05-10 04:55:17,002 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-05-10 04:55:17,002 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:55:17,002 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-05-10 04:55:17,003 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-05-10 04:55:17,003 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:17,003 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-05-10 04:55:17,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:17,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:55:17,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:55:17,003 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:55:17,003 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:55:17,003 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-05-10 04:55:17,003 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-05-10 04:55:17,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:55:17,003 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:55:17,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-10 04:55:17,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:17,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:17,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:17,004 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:55:17,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-10 04:55:17,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:17,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:17,005 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:17,006 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,36665,1683694419452': 2023-05-10 04:55:17,006 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36665,1683694419452' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-05-10 04:55:17,006 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-05-10 04:55:17,006 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-10 04:55:17,006 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-10 04:55:17,006 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-05-10 04:55:17,006 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-10 04:55:17,009 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,009 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,009 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,009 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:17,009 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,009 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:17,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,009 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:55:17,009 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:17,009 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:55:17,009 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,009 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:17,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,009 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:17,010 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:17,010 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:17,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:17,010 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:17,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:17,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:55:17,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,010 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:17,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,010 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-10 04:55:17,010 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:17,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:55:17,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-10 04:55:17,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:17,011 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:17,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:17,012 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:17,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:55:17,012 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:17,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-10 04:55:17,012 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:17,012 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:17,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:17,013 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:17,015 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:17,015 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:17,015 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:17,015 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:17,015 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:17,015 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:55:17,015 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:17,015 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-05-10 04:55:17,015 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,015 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-10 04:55:17,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:17,015 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:17,015 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:17,015 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-05-10 04:55:17,015 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:17,015 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:17,015 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:17,016 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:17,016 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-10 04:55:17,015 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:17,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:17,015 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:17,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:17,016 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:17,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:17,016 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,016 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:17,016 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,016 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:17,016 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:17,016 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,016 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,016 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:17,016 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:17,116 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-10 04:55:17,117 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-10 04:55:17,127 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-10 04:55:17,129 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-05-10 04:55:17,129 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-05-10 04:55:17,129 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-05-10 04:55:17,129 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:55:17,129 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-05-10 04:55:17,129 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-05-10 04:55:17,129 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,129 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,130 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:17,130 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:17,130 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:17,130 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:17,130 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:17,130 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:17,130 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:17,131 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:17,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:17,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:17,131 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:17,131 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-10 04:55:17,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,131 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,131 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,131 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,131 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:55:17,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:55:17,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:55:17,132 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,132 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-05-10 04:55:17,132 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-05-10 04:55:17,132 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-05-10 04:55:17,132 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-05-10 04:55:17,132 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-05-10 04:55:17,132 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-05-10 04:55:17,132 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-05-10 04:55:17,132 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:55:17,132 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-05-10 04:55:17,132 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-05-10 04:55:17,132 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-05-10 04:55:17,132 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-05-10 04:55:17,132 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-05-10 04:55:17,132 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:55:17,132 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-05-10 04:55:17,133 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-05-10 04:55:17,132 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37995,1683694419499' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-05-10 04:55:17,133 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-05-10 04:55:17,133 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:55:17,133 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-05-10 04:55:17,133 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-05-10 04:55:17,133 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37823,1683694419534' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-05-10 04:55:17,133 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-10 04:55:17,134 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-10 04:55:17,134 DEBUG [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1683694516443.ba55ae97fe6d26f2fd2a296837570536. 2023-05-10 04:55:17,134 DEBUG [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1683694516443.ba55ae97fe6d26f2fd2a296837570536. started... 2023-05-10 04:55:17,134 INFO [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing ba55ae97fe6d26f2fd2a296837570536 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-10 04:55:17,134 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,136 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,136 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,136 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:55:17,136 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,136 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:55:17,143 INFO [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/.tmp/cf1/3488050453f04078bc1365497a2e3f88 2023-05-10 04:55:17,153 INFO [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/.tmp/cf2/1747520752eb459e951b7491aa7c62a3 2023-05-10 04:55:17,156 DEBUG [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/.tmp/cf1/3488050453f04078bc1365497a2e3f88 as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf1/3488050453f04078bc1365497a2e3f88 2023-05-10 04:55:17,160 INFO [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf1/3488050453f04078bc1365497a2e3f88, entries=10, sequenceid=19, filesize=5.0 K 2023-05-10 04:55:17,160 DEBUG [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/.tmp/cf2/1747520752eb459e951b7491aa7c62a3 as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf2/1747520752eb459e951b7491aa7c62a3 2023-05-10 04:55:17,163 INFO [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf2/1747520752eb459e951b7491aa7c62a3, entries=10, sequenceid=19, filesize=5.0 K 2023-05-10 04:55:17,164 INFO [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for ba55ae97fe6d26f2fd2a296837570536 in 30ms, sequenceid=19, compaction requested=false 2023-05-10 04:55:17,164 DEBUG [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for ba55ae97fe6d26f2fd2a296837570536: 2023-05-10 04:55:17,164 DEBUG [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1683694516443.ba55ae97fe6d26f2fd2a296837570536. 2023-05-10 04:55:17,164 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-10 04:55:17,164 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-10 04:55:17,164 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:17,165 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-05-10 04:55:17,165 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36665,1683694419452' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-05-10 04:55:17,174 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:17,174 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:17,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:55:17,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:55:17,174 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,174 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:55:17,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:55:17,174 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:55:17,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-10 04:55:17,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:17,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:17,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:17,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:55:17,176 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36665,1683694419452' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-05-10 04:55:17,176 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@66e2de1f[Count = 0] remaining members to acquire global barrier 2023-05-10 04:55:17,176 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-05-10 04:55:17,177 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,178 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,178 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,178 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,178 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,178 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:17,178 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,178 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,178 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-05-10 04:55:17,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,178 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-05-10 04:55:17,178 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-05-10 04:55:17,178 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-05-10 04:55:17,178 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,37823,1683694419534' in zk 2023-05-10 04:55:17,178 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-10 04:55:17,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,178 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,37995,1683694419499' in zk 2023-05-10 04:55:17,178 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-05-10 04:55:17,179 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-05-10 04:55:17,179 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,36665,1683694419452' in zk 2023-05-10 04:55:17,180 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-05-10 04:55:17,180 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:55:17,180 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-05-10 04:55:17,182 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-05-10 04:55:17,182 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:17,182 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-05-10 04:55:17,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:17,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:55:17,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:55:17,182 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:55:17,182 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:55:17,182 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-05-10 04:55:17,182 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-05-10 04:55:17,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:55:17,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:55:17,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-10 04:55:17,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:17,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:17,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:17,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:55:17,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-10 04:55:17,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:17,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:17,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:17,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,36665,1683694419452': 2023-05-10 04:55:17,185 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36665,1683694419452' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-05-10 04:55:17,185 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-05-10 04:55:17,185 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-10 04:55:17,185 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-10 04:55:17,185 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-05-10 04:55:17,185 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-10 04:55:17,187 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,187 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,187 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:17,187 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,187 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,187 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,187 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,187 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,187 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:17,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,187 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:17,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,187 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:17,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:55:17,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:55:17,187 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:17,187 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:17,187 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:17,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:17,188 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:17,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:17,188 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:17,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:55:17,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,188 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:17,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-10 04:55:17,188 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:55:17,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-10 04:55:17,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:17,189 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:17,190 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:17,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:17,190 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:17,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:55:17,190 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:17,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-10 04:55:17,190 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:17,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:17,191 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:17,192 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:17,192 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:17,192 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:17,192 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:17,192 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:17,193 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-05-10 04:55:17,193 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-10 04:55:17,192 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,193 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:17,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:17,193 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:17,193 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:55:17,193 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:17,193 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-05-10 04:55:17,193 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-10 04:55:17,193 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:17,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:17,192 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:17,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:17,193 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:17,193 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:17,193 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:17,193 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,193 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:17,193 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:17,194 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:17,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:17,193 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:17,194 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:17,194 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,194 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:17,194 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:17,194 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,194 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,293 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-10 04:55:17,294 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-10 04:55:17,305 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-10 04:55:17,306 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-05-10 04:55:17,307 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-05-10 04:55:17,307 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-05-10 04:55:17,307 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:55:17,307 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-05-10 04:55:17,307 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-05-10 04:55:17,307 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,307 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,308 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:17,308 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:17,308 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:17,308 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:17,308 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:17,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:17,308 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:17,308 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:17,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:17,309 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:17,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:17,309 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-10 04:55:17,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,309 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,309 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:55:17,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,309 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,309 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-05-10 04:55:17,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:55:17,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,310 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-05-10 04:55:17,310 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-05-10 04:55:17,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:55:17,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,310 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-05-10 04:55:17,310 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-05-10 04:55:17,310 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-05-10 04:55:17,310 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-05-10 04:55:17,310 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-05-10 04:55:17,310 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-05-10 04:55:17,310 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:55:17,310 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:55:17,310 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-05-10 04:55:17,310 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-05-10 04:55:17,310 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-05-10 04:55:17,310 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37995,1683694419499' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-05-10 04:55:17,310 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-05-10 04:55:17,310 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-05-10 04:55:17,310 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-10 04:55:17,310 DEBUG [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1683694516443.ba55ae97fe6d26f2fd2a296837570536. 2023-05-10 04:55:17,310 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-10 04:55:17,310 DEBUG [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1683694516443.ba55ae97fe6d26f2fd2a296837570536. started... 2023-05-10 04:55:17,310 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-05-10 04:55:17,311 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:55:17,311 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-05-10 04:55:17,311 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-05-10 04:55:17,311 INFO [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing ba55ae97fe6d26f2fd2a296837570536 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-10 04:55:17,311 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37823,1683694419534' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-05-10 04:55:17,312 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,313 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,313 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:55:17,313 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,314 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,314 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:55:17,319 INFO [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/.tmp/cf1/c2d599f271fb497b8b272a3bfd391f6e 2023-05-10 04:55:17,330 INFO [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/.tmp/cf2/c332f2f6f5a44943a5e8c97725d45512 2023-05-10 04:55:17,333 DEBUG [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/.tmp/cf1/c2d599f271fb497b8b272a3bfd391f6e as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf1/c2d599f271fb497b8b272a3bfd391f6e 2023-05-10 04:55:17,337 INFO [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf1/c2d599f271fb497b8b272a3bfd391f6e, entries=10, sequenceid=32, filesize=5.0 K 2023-05-10 04:55:17,338 DEBUG [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/.tmp/cf2/c332f2f6f5a44943a5e8c97725d45512 as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf2/c332f2f6f5a44943a5e8c97725d45512 2023-05-10 04:55:17,341 INFO [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf2/c332f2f6f5a44943a5e8c97725d45512, entries=10, sequenceid=32, filesize=5.0 K 2023-05-10 04:55:17,342 INFO [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for ba55ae97fe6d26f2fd2a296837570536 in 31ms, sequenceid=32, compaction requested=true 2023-05-10 04:55:17,342 DEBUG [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for ba55ae97fe6d26f2fd2a296837570536: 2023-05-10 04:55:17,342 DEBUG [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1683694516443.ba55ae97fe6d26f2fd2a296837570536. 2023-05-10 04:55:17,342 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-10 04:55:17,342 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-10 04:55:17,342 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:17,342 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-05-10 04:55:17,342 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36665,1683694419452' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-05-10 04:55:17,343 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,343 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:17,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:17,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:55:17,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:55:17,344 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,344 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:55:17,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:55:17,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:55:17,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-10 04:55:17,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:17,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:17,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:17,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:55:17,346 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36665,1683694419452' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-05-10 04:55:17,346 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@56802c8b[Count = 0] remaining members to acquire global barrier 2023-05-10 04:55:17,346 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-05-10 04:55:17,346 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,347 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,347 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,347 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,347 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,347 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,347 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-05-10 04:55:17,347 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,347 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-05-10 04:55:17,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,347 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,36665,1683694419452' in zk 2023-05-10 04:55:17,348 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-05-10 04:55:17,348 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-05-10 04:55:17,347 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:17,348 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-10 04:55:17,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,348 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,37995,1683694419499' in zk 2023-05-10 04:55:17,348 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-05-10 04:55:17,348 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-05-10 04:55:17,348 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,37823,1683694419534' in zk 2023-05-10 04:55:17,350 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-05-10 04:55:17,350 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:17,350 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:55:17,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:17,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:55:17,350 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-05-10 04:55:17,351 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:55:17,351 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-05-10 04:55:17,351 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:55:17,351 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-05-10 04:55:17,352 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-05-10 04:55:17,352 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:55:17,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:55:17,352 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-05-10 04:55:17,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:55:17,352 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-10 04:55:17,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:17,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:17,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:17,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:55:17,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-10 04:55:17,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:17,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:17,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:17,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,36665,1683694419452': 2023-05-10 04:55:17,355 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36665,1683694419452' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-05-10 04:55:17,355 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-05-10 04:55:17,355 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-10 04:55:17,355 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-10 04:55:17,355 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-05-10 04:55:17,355 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-10 04:55:17,356 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,357 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,357 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:17,357 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:17,357 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,357 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,357 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:17,357 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,357 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:55:17,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:55:17,357 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,357 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:17,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:17,357 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:17,357 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:17,357 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:17,358 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:17,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:17,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:55:17,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,358 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:17,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-10 04:55:17,358 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:17,358 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:55:17,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-10 04:55:17,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:17,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:17,359 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:17,360 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:17,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:55:17,360 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:17,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-10 04:55:17,360 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:17,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:17,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:17,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:17,366 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:17,367 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:17,367 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:17,367 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:17,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:17,367 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:17,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:17,367 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:17,367 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:17,367 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,367 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:17,367 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-05-10 04:55:17,367 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-10 04:55:17,367 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:17,367 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:55:17,367 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:17,367 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:17,367 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:17,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:17,367 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:17,367 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:17,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:17,367 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,367 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:17,367 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-05-10 04:55:17,368 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:17,368 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:17,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:17,368 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-10 04:55:17,368 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,368 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:17,368 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:17,368 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,368 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,468 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-10 04:55:17,469 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-10 04:55:17,479 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-10 04:55:17,481 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-05-10 04:55:17,481 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-05-10 04:55:17,481 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-05-10 04:55:17,481 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:55:17,481 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-05-10 04:55:17,481 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-05-10 04:55:17,481 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,481 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,483 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:17,483 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:17,483 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:17,483 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:17,483 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:17,483 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:17,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:17,483 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:17,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:17,483 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:17,483 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-10 04:55:17,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:17,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,483 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,483 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,484 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,484 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:55:17,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,484 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-05-10 04:55:17,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:55:17,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:55:17,484 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-05-10 04:55:17,484 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,484 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-05-10 04:55:17,484 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-05-10 04:55:17,484 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-05-10 04:55:17,484 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-05-10 04:55:17,484 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-05-10 04:55:17,484 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:55:17,484 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-05-10 04:55:17,484 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-05-10 04:55:17,484 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-05-10 04:55:17,485 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:55:17,484 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-05-10 04:55:17,485 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-05-10 04:55:17,485 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-05-10 04:55:17,484 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-05-10 04:55:17,485 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37995,1683694419499' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-05-10 04:55:17,485 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-05-10 04:55:17,485 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:55:17,485 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-05-10 04:55:17,485 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37823,1683694419534' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-05-10 04:55:17,485 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-05-10 04:55:17,485 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-10 04:55:17,485 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-10 04:55:17,485 DEBUG [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1683694516443.ba55ae97fe6d26f2fd2a296837570536. 2023-05-10 04:55:17,485 DEBUG [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1683694516443.ba55ae97fe6d26f2fd2a296837570536. started... 2023-05-10 04:55:17,485 INFO [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing ba55ae97fe6d26f2fd2a296837570536 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-10 04:55:17,486 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,488 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,488 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,488 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:55:17,488 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,488 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:55:17,494 INFO [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/.tmp/cf1/bd5a17faa7564deb857bb9ec28719ee6 2023-05-10 04:55:17,504 INFO [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/.tmp/cf2/c845912c71644277a0aed468d0621d59 2023-05-10 04:55:17,508 DEBUG [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/.tmp/cf1/bd5a17faa7564deb857bb9ec28719ee6 as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf1/bd5a17faa7564deb857bb9ec28719ee6 2023-05-10 04:55:17,512 INFO [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf1/bd5a17faa7564deb857bb9ec28719ee6, entries=10, sequenceid=45, filesize=5.0 K 2023-05-10 04:55:17,512 DEBUG [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/.tmp/cf2/c845912c71644277a0aed468d0621d59 as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf2/c845912c71644277a0aed468d0621d59 2023-05-10 04:55:17,516 INFO [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf2/c845912c71644277a0aed468d0621d59, entries=10, sequenceid=45, filesize=5.0 K 2023-05-10 04:55:17,516 INFO [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for ba55ae97fe6d26f2fd2a296837570536 in 31ms, sequenceid=45, compaction requested=true 2023-05-10 04:55:17,517 DEBUG [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for ba55ae97fe6d26f2fd2a296837570536: 2023-05-10 04:55:17,517 DEBUG [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1683694516443.ba55ae97fe6d26f2fd2a296837570536. 2023-05-10 04:55:17,517 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-10 04:55:17,517 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-10 04:55:17,517 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:17,517 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-05-10 04:55:17,517 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36665,1683694419452' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-05-10 04:55:17,518 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:17,518 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:17,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:55:17,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:55:17,519 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,519 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:55:17,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:55:17,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:55:17,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-10 04:55:17,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:17,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:17,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:17,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:55:17,520 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36665,1683694419452' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-05-10 04:55:17,520 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5cb3588[Count = 0] remaining members to acquire global barrier 2023-05-10 04:55:17,520 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-05-10 04:55:17,520 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,522 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,522 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,522 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,522 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,522 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,522 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,522 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:17,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,522 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-10 04:55:17,522 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-05-10 04:55:17,522 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-05-10 04:55:17,522 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-05-10 04:55:17,522 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-05-10 04:55:17,522 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,37823,1683694419534' in zk 2023-05-10 04:55:17,522 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-05-10 04:55:17,522 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-05-10 04:55:17,522 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,37995,1683694419499' in zk 2023-05-10 04:55:17,522 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,36665,1683694419452' in zk 2023-05-10 04:55:17,523 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-05-10 04:55:17,523 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-05-10 04:55:17,523 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:17,523 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-05-10 04:55:17,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:17,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:55:17,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:55:17,523 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:55:17,523 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:55:17,524 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-05-10 04:55:17,524 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:55:17,524 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-05-10 04:55:17,524 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-05-10 04:55:17,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:55:17,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:55:17,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-10 04:55:17,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:17,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:17,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:17,525 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:55:17,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-10 04:55:17,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:17,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:17,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:17,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,36665,1683694419452': 2023-05-10 04:55:17,527 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36665,1683694419452' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-05-10 04:55:17,527 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-05-10 04:55:17,527 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-10 04:55:17,527 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-10 04:55:17,527 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-05-10 04:55:17,527 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-10 04:55:17,529 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,529 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,529 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,529 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:17,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:55:17,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:55:17,529 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,529 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,529 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,529 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:17,529 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:17,529 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:55:17,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,529 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:17,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:17,529 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:17,530 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:17,530 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:17,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:17,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-10 04:55:17,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:17,530 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:17,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:55:17,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,530 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:17,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-10 04:55:17,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:17,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:17,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:17,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:55:17,532 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:17,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-10 04:55:17,532 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:17,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:17,532 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:17,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:17,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:17,534 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:17,535 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:17,535 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:17,535 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:17,535 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:17,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:17,535 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:17,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:17,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:17,538 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:17,538 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,538 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:17,539 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:17,539 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,539 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:17,539 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:17,539 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:17,539 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:17,539 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,539 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:17,539 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:17,539 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,539 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,539 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:17,539 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:17,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:17,539 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-05-10 04:55:17,539 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-10 04:55:17,539 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:17,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:17,539 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:17,539 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:55:17,540 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-05-10 04:55:17,540 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-10 04:55:17,640 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-10 04:55:17,641 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-10 04:55:17,651 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-10 04:55:17,652 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-05-10 04:55:17,652 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-05-10 04:55:17,652 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-05-10 04:55:17,653 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:55:17,653 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-05-10 04:55:17,653 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-05-10 04:55:17,653 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,653 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,654 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:17,654 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:17,654 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:17,654 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:17,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:17,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:17,654 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:17,654 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:17,654 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:17,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:17,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,655 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:17,655 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-10 04:55:17,655 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,655 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,655 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:55:17,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,655 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-05-10 04:55:17,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:55:17,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:55:17,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,656 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-05-10 04:55:17,656 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-05-10 04:55:17,656 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-05-10 04:55:17,656 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-05-10 04:55:17,656 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-05-10 04:55:17,656 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:55:17,656 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-05-10 04:55:17,656 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-05-10 04:55:17,656 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-05-10 04:55:17,656 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37823,1683694419534' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-05-10 04:55:17,656 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-05-10 04:55:17,656 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-05-10 04:55:17,656 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-05-10 04:55:17,656 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:55:17,656 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-05-10 04:55:17,656 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-05-10 04:55:17,656 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-05-10 04:55:17,656 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-05-10 04:55:17,656 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37995,1683694419499' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-05-10 04:55:17,656 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:55:17,656 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-05-10 04:55:17,657 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-10 04:55:17,657 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-10 04:55:17,657 DEBUG [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1683694516443.ba55ae97fe6d26f2fd2a296837570536. 2023-05-10 04:55:17,657 DEBUG [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1683694516443.ba55ae97fe6d26f2fd2a296837570536. started... 2023-05-10 04:55:17,657 INFO [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing ba55ae97fe6d26f2fd2a296837570536 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-10 04:55:17,657 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,658 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,658 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:55:17,659 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,660 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,660 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:55:17,665 INFO [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/.tmp/cf1/b3e182af00c84ddb81eb7331183b802e 2023-05-10 04:55:17,676 INFO [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/.tmp/cf2/14e73a6c8cd44ed2ab40f783dad882ca 2023-05-10 04:55:17,680 DEBUG [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/.tmp/cf1/b3e182af00c84ddb81eb7331183b802e as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf1/b3e182af00c84ddb81eb7331183b802e 2023-05-10 04:55:17,683 INFO [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf1/b3e182af00c84ddb81eb7331183b802e, entries=10, sequenceid=58, filesize=5.0 K 2023-05-10 04:55:17,684 DEBUG [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/.tmp/cf2/14e73a6c8cd44ed2ab40f783dad882ca as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf2/14e73a6c8cd44ed2ab40f783dad882ca 2023-05-10 04:55:17,687 INFO [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf2/14e73a6c8cd44ed2ab40f783dad882ca, entries=10, sequenceid=58, filesize=5.0 K 2023-05-10 04:55:17,687 INFO [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for ba55ae97fe6d26f2fd2a296837570536 in 30ms, sequenceid=58, compaction requested=true 2023-05-10 04:55:17,688 DEBUG [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for ba55ae97fe6d26f2fd2a296837570536: 2023-05-10 04:55:17,688 DEBUG [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1683694516443.ba55ae97fe6d26f2fd2a296837570536. 2023-05-10 04:55:17,688 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-10 04:55:17,688 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-10 04:55:17,688 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:17,688 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-05-10 04:55:17,688 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36665,1683694419452' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-05-10 04:55:17,691 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,691 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:17,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:17,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:55:17,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:55:17,691 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,692 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:55:17,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:55:17,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:55:17,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-10 04:55:17,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:17,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:17,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:17,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:55:17,693 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36665,1683694419452' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-05-10 04:55:17,693 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@31000209[Count = 0] remaining members to acquire global barrier 2023-05-10 04:55:17,693 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-05-10 04:55:17,693 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,694 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,694 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,694 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,694 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,694 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,694 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-05-10 04:55:17,695 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:17,695 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-10 04:55:17,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,695 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-05-10 04:55:17,695 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-05-10 04:55:17,695 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-05-10 04:55:17,695 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,37995,1683694419499' in zk 2023-05-10 04:55:17,695 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,695 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,37823,1683694419534' in zk 2023-05-10 04:55:17,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,695 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-05-10 04:55:17,695 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-05-10 04:55:17,695 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,36665,1683694419452' in zk 2023-05-10 04:55:17,696 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-05-10 04:55:17,696 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-05-10 04:55:17,696 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:55:17,696 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:55:17,696 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-05-10 04:55:17,696 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-05-10 04:55:17,698 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-05-10 04:55:17,698 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:17,698 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:55:17,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:17,698 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-05-10 04:55:17,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:55:17,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:55:17,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:55:17,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:55:17,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-10 04:55:17,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:17,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:17,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:17,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:55:17,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-10 04:55:17,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:17,700 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:17,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:17,701 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,36665,1683694419452': 2023-05-10 04:55:17,701 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36665,1683694419452' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-05-10 04:55:17,701 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-05-10 04:55:17,701 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-10 04:55:17,701 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-10 04:55:17,701 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-05-10 04:55:17,701 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-10 04:55:17,703 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,703 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,703 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,703 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,703 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,703 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,703 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:17,703 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:17,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:55:17,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:55:17,703 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:17,703 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:17,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,703 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,703 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:17,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:55:17,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:17,703 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:17,703 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:17,704 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:17,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:17,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:17,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-10 04:55:17,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,704 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:17,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:55:17,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-10 04:55:17,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:17,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:17,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:17,705 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:17,705 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:55:17,706 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:17,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-10 04:55:17,706 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:17,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:17,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:17,706 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:17,708 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:17,708 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,708 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:17,708 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:17,708 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:17,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:17,708 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:17,708 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:17,708 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:17,708 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:55:17,708 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-05-10 04:55:17,709 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-10 04:55:17,709 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-05-10 04:55:17,709 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-10 04:55:17,708 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:17,708 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:17,708 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:17,708 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:17,708 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:17,708 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:17,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:17,710 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:17,710 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:17,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:17,710 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:17,710 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:17,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:17,710 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:17,711 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,711 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:17,711 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:17,711 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,711 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,809 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-10 04:55:17,810 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-10 04:55:17,821 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-10 04:55:17,822 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-05-10 04:55:17,822 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-05-10 04:55:17,822 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-05-10 04:55:17,822 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:55:17,822 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-05-10 04:55:17,823 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-05-10 04:55:17,823 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,823 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,825 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:17,825 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:17,825 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:17,825 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:17,825 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:17,825 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:17,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:17,825 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:17,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:17,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:17,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,825 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:17,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,826 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,825 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-10 04:55:17,826 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,826 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:55:17,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,826 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-05-10 04:55:17,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:55:17,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:55:17,826 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,826 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-05-10 04:55:17,826 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-05-10 04:55:17,826 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-05-10 04:55:17,826 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-05-10 04:55:17,826 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-05-10 04:55:17,826 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:55:17,826 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-05-10 04:55:17,826 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-05-10 04:55:17,826 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-05-10 04:55:17,827 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-05-10 04:55:17,827 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-05-10 04:55:17,827 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-05-10 04:55:17,827 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-05-10 04:55:17,827 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-05-10 04:55:17,827 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:55:17,826 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37823,1683694419534' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-05-10 04:55:17,827 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-05-10 04:55:17,827 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:55:17,827 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-05-10 04:55:17,827 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-05-10 04:55:17,827 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-10 04:55:17,827 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-10 04:55:17,827 DEBUG [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1683694516443.ba55ae97fe6d26f2fd2a296837570536. 2023-05-10 04:55:17,827 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37995,1683694419499' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-05-10 04:55:17,827 DEBUG [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1683694516443.ba55ae97fe6d26f2fd2a296837570536. started... 2023-05-10 04:55:17,827 INFO [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing ba55ae97fe6d26f2fd2a296837570536 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-10 04:55:17,830 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,831 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,831 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,831 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:55:17,831 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,831 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:55:17,836 INFO [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/.tmp/cf1/aa810537cf12426fa1bcdd14225b68ee 2023-05-10 04:55:17,847 INFO [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/.tmp/cf2/c4912c4098564070a195e3fd918ecec2 2023-05-10 04:55:17,850 DEBUG [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/.tmp/cf1/aa810537cf12426fa1bcdd14225b68ee as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf1/aa810537cf12426fa1bcdd14225b68ee 2023-05-10 04:55:17,853 INFO [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf1/aa810537cf12426fa1bcdd14225b68ee, entries=10, sequenceid=71, filesize=5.0 K 2023-05-10 04:55:17,854 DEBUG [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/.tmp/cf2/c4912c4098564070a195e3fd918ecec2 as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf2/c4912c4098564070a195e3fd918ecec2 2023-05-10 04:55:17,857 INFO [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf2/c4912c4098564070a195e3fd918ecec2, entries=10, sequenceid=71, filesize=5.0 K 2023-05-10 04:55:17,858 INFO [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for ba55ae97fe6d26f2fd2a296837570536 in 31ms, sequenceid=71, compaction requested=true 2023-05-10 04:55:17,858 DEBUG [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for ba55ae97fe6d26f2fd2a296837570536: 2023-05-10 04:55:17,858 DEBUG [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1683694516443.ba55ae97fe6d26f2fd2a296837570536. 2023-05-10 04:55:17,858 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-10 04:55:17,858 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-10 04:55:17,858 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:17,858 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-05-10 04:55:17,858 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36665,1683694419452' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-05-10 04:55:17,860 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:17,860 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:17,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:55:17,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:55:17,860 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,860 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:55:17,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:55:17,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:55:17,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-10 04:55:17,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:17,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:17,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:17,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:55:17,862 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36665,1683694419452' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-05-10 04:55:17,862 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@41e23010[Count = 0] remaining members to acquire global barrier 2023-05-10 04:55:17,862 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-05-10 04:55:17,862 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,863 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,863 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,863 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,863 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,863 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,863 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-05-10 04:55:17,863 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,863 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-05-10 04:55:17,863 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-05-10 04:55:17,863 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:17,863 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-10 04:55:17,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,863 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-05-10 04:55:17,864 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-05-10 04:55:17,863 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,37823,1683694419534' in zk 2023-05-10 04:55:17,864 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-05-10 04:55:17,864 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,36665,1683694419452' in zk 2023-05-10 04:55:17,864 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase4.apache.org,37995,1683694419499' in zk 2023-05-10 04:55:17,866 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-05-10 04:55:17,866 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:55:17,866 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:17,866 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-05-10 04:55:17,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:17,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:55:17,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:55:17,866 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-05-10 04:55:17,866 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:55:17,866 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-05-10 04:55:17,866 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-05-10 04:55:17,866 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:55:17,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:55:17,866 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-05-10 04:55:17,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:55:17,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-10 04:55:17,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:17,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:17,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:17,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:55:17,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-10 04:55:17,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:17,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:17,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:17,869 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase4.apache.org,36665,1683694419452': 2023-05-10 04:55:17,869 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36665,1683694419452' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-05-10 04:55:17,869 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-05-10 04:55:17,869 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-10 04:55:17,869 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-10 04:55:17,869 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-05-10 04:55:17,869 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-10 04:55:17,870 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,870 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,870 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,870 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,870 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,870 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,870 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:17,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:55:17,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:55:17,870 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:17,870 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:17,871 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:17,871 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:17,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:55:17,870 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:17,871 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:17,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:17,871 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:17,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-10 04:55:17,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,871 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:17,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:17,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,872 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:17,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:55:17,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-10 04:55:17,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:17,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:17,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:17,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:55:17,873 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:17,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-05-10 04:55:17,873 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:17,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:17,874 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:17,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:17,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:17,880 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:17,880 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,880 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:17,880 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:17,880 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:17,880 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:17,880 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:17,880 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:17,880 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:55:17,880 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:17,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:17,880 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:17,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:17,880 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,880 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:17,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:17,880 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:17,880 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:17,880 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:17,880 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-05-10 04:55:17,881 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:17,881 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-10 04:55:17,881 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:17,881 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:17,881 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:17,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:17,881 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:17,881 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-05-10 04:55:17,881 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:17,881 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-10 04:55:17,881 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:17,881 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,881 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-05-10 04:55:17,981 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-10 04:55:17,983 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-10 04:55:18,240 WARN [Listener at localhost/45007] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: true 2023-05-10 04:55:18,241 DEBUG [Listener at localhost/45007] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:18,242 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(1029): Stored pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ba55ae97fe6d26f2fd2a296837570536, daughterA=48042d1ae86f00fdead71e71ad8292a9, daughterB=1b26b3dca87c03702b1dc11207d7b72a 2023-05-10 04:55:18,243 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ba55ae97fe6d26f2fd2a296837570536, daughterA=48042d1ae86f00fdead71e71ad8292a9, daughterB=1b26b3dca87c03702b1dc11207d7b72a 2023-05-10 04:55:18,243 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ba55ae97fe6d26f2fd2a296837570536, daughterA=48042d1ae86f00fdead71e71ad8292a9, daughterB=1b26b3dca87c03702b1dc11207d7b72a 2023-05-10 04:55:18,243 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ba55ae97fe6d26f2fd2a296837570536, daughterA=48042d1ae86f00fdead71e71ad8292a9, daughterB=1b26b3dca87c03702b1dc11207d7b72a 2023-05-10 04:55:18,244 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-10 04:55:18,244 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ba55ae97fe6d26f2fd2a296837570536, daughterA=48042d1ae86f00fdead71e71ad8292a9, daughterB=1b26b3dca87c03702b1dc11207d7b72a 2023-05-10 04:55:18,244 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-10 04:55:18,244 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ba55ae97fe6d26f2fd2a296837570536, daughterA=48042d1ae86f00fdead71e71ad8292a9, daughterB=1b26b3dca87c03702b1dc11207d7b72a at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-10 04:55:18,492 INFO [Listener at localhost/45007] procedure.MasterProcedureTestingUtility(454): Restart 0 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ba55ae97fe6d26f2fd2a296837570536, daughterA=48042d1ae86f00fdead71e71ad8292a9, daughterB=1b26b3dca87c03702b1dc11207d7b72a 2023-05-10 04:55:18,493 INFO [Listener at localhost/45007] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-10 04:55:18,493 INFO [Listener at localhost/45007] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-10 04:55:18,493 INFO [Listener at localhost/45007] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-10 04:55:18,493 ERROR [Listener at localhost/45007] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-9,5,PEWorkerGroup] Thread[HFileArchiver-10,5,PEWorkerGroup] Thread[HFileArchiver-11,5,PEWorkerGroup] Thread[HFileArchiver-12,5,PEWorkerGroup] 2023-05-10 04:55:18,493 INFO [Listener at localhost/45007] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-10 04:55:18,493 INFO [Listener at localhost/45007] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-10 04:55:18,493 INFO [Listener at localhost/45007] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-10 04:55:18,493 INFO [Listener at localhost/45007] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-10 04:55:18,494 INFO [Listener at localhost/45007] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-10 04:55:18,495 DEBUG [Listener at localhost/45007] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[] 2023-05-10 04:55:18,495 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-10 04:55:18,496 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-10 04:55:18,496 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-10 04:55:18,496 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-10 04:55:18,496 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:55:18,497 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=93c2cc329b1fe1893cfbc28c56960522, daughterA=19040b6538a6bdeed7c833cfb44614b6, daughterB=054f69c06f126aad97b290410d5d6188 2023-05-10 04:55:18,497 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:55:18,497 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:55:18,497 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-10 04:55:18,497 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6c70954228a0bbe97dfdc76861367c66, daughterA=75e2592cb2ac08bd9f617bb9d0a0c896, daughterB=b90fd344aff14f0728a80c3d09539e54 2023-05-10 04:55:18,497 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-05-10 04:55:18,497 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-10 04:55:18,498 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-10 04:55:18,498 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:55:18,498 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=4dcb62249031b560a1bc5d03e43ac855, ASSIGN 2023-05-10 04:55:18,498 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-05-10 04:55:18,498 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-05-10 04:55:18,498 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-10 04:55:18,499 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=a7ae61c05a799883e33cfd9dfad56fa1, daughterA=944df92d059326a46982ff7979f8a940, daughterB=fa1eb74e4c5521990fb9118cc6dfd407 2023-05-10 04:55:18,499 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-10 04:55:18,499 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-10 04:55:18,499 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:55:18,499 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=e0a753dfc7b1edf68920ae1543edc38c, daughterA=08557250442eebc7c86235cdf682b1a9, daughterB=e1dcbf091d2db6dbef8d701eb3d89914 2023-05-10 04:55:18,499 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:55:18,499 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:55:18,500 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-10 04:55:18,500 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-10 04:55:18,500 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,37995,1683694419499, table=testRollbackForSplitTableRegionWithReplica, region=bddd3bfc356670bc9cd9492c47d95a01 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=d2b34babccb97d3c1ac066d9e90ae654, daughterA=0141d5003cd2aac0ddef526f1a98709b, daughterB=b919ef6ce9c770ae26c6c5ee0d2ac459 2023-05-10 04:55:18,500 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-10 04:55:18,500 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-10 04:55:18,501 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-05-10 04:55:18,501 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-05-10 04:55:18,501 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-05-10 04:55:18,501 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-05-10 04:55:18,501 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ba55ae97fe6d26f2fd2a296837570536, daughterA=48042d1ae86f00fdead71e71ad8292a9, daughterB=1b26b3dca87c03702b1dc11207d7b72a 2023-05-10 04:55:18,501 DEBUG [Listener at localhost/45007] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ba55ae97fe6d26f2fd2a296837570536, daughterA=48042d1ae86f00fdead71e71ad8292a9, daughterB=1b26b3dca87c03702b1dc11207d7b72a held the lock before restarting, call acquireLock to restore it. 2023-05-10 04:55:18,501 INFO [Listener at localhost/45007] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ba55ae97fe6d26f2fd2a296837570536, daughterA=48042d1ae86f00fdead71e71ad8292a9, daughterB=1b26b3dca87c03702b1dc11207d7b72a 2023-05-10 04:55:18,501 INFO [Listener at localhost/45007] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ba55ae97fe6d26f2fd2a296837570536, daughterA=48042d1ae86f00fdead71e71ad8292a9, daughterB=1b26b3dca87c03702b1dc11207d7b72a 2023-05-10 04:55:18,501 INFO [Listener at localhost/45007] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ba55ae97fe6d26f2fd2a296837570536, daughterA=48042d1ae86f00fdead71e71ad8292a9, daughterB=1b26b3dca87c03702b1dc11207d7b72a 2023-05-10 04:55:18,502 INFO [Listener at localhost/45007] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-05-10 04:55:18,502 DEBUG [Listener at localhost/45007] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-10 04:55:18,503 DEBUG [Listener at localhost/45007] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,37995,1683694419499, table=hbase:meta, region=1588230740 2023-05-10 04:55:18,503 INFO [Listener at localhost/45007] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683694548503 2023-05-10 04:55:18,505 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ba55ae97fe6d26f2fd2a296837570536, daughterA=48042d1ae86f00fdead71e71ad8292a9, daughterB=1b26b3dca87c03702b1dc11207d7b72a 2023-05-10 04:55:18,505 DEBUG [Listener at localhost/45007] assignment.AssignmentManager(1517): Joining cluster... 2023-05-10 04:55:18,507 DEBUG [Listener at localhost/45007] assignment.RegionStateStore(147): Load hbase:meta entry region=ccfda81dc2f13ffc1429bef67d349028, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36665,1683694419452, regionLocation=jenkins-hbase4.apache.org,36665,1683694419452, openSeqNum=2 2023-05-10 04:55:18,507 DEBUG [Listener at localhost/45007] assignment.RegionStateStore(147): Load hbase:meta entry region=ba55ae97fe6d26f2fd2a296837570536, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36665,1683694419452, regionLocation=jenkins-hbase4.apache.org,36665,1683694419452, openSeqNum=2 2023-05-10 04:55:18,507 INFO [Listener at localhost/45007] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-10 04:55:18,507 INFO [Listener at localhost/45007] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683694578507 2023-05-10 04:55:18,507 INFO [Listener at localhost/45007] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683694638507 2023-05-10 04:55:18,507 INFO [Listener at localhost/45007] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-05-10 04:55:18,507 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-10 04:55:18,509 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-10 04:55:18,509 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ba55ae97fe6d26f2fd2a296837570536, daughterA=48042d1ae86f00fdead71e71ad8292a9, daughterB=1b26b3dca87c03702b1dc11207d7b72a 2023-05-10 04:55:18,509 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-10 04:55:18,509 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ba55ae97fe6d26f2fd2a296837570536, daughterA=48042d1ae86f00fdead71e71ad8292a9, daughterB=1b26b3dca87c03702b1dc11207d7b72a at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-10 04:55:18,757 INFO [Listener at localhost/45007] procedure.MasterProcedureTestingUtility(454): Restart 1 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ba55ae97fe6d26f2fd2a296837570536, daughterA=48042d1ae86f00fdead71e71ad8292a9, daughterB=1b26b3dca87c03702b1dc11207d7b72a 2023-05-10 04:55:18,757 INFO [Listener at localhost/45007] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-10 04:55:18,757 INFO [Listener at localhost/45007] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-10 04:55:18,758 INFO [Listener at localhost/45007] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-10 04:55:18,758 ERROR [Listener at localhost/45007] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-10 04:55:18,758 INFO [Listener at localhost/45007] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-10 04:55:18,758 INFO [Listener at localhost/45007] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-10 04:55:18,758 INFO [Listener at localhost/45007] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-10 04:55:18,758 INFO [Listener at localhost/45007] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-10 04:55:18,759 INFO [Listener at localhost/45007] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-10 04:55:18,760 DEBUG [Listener at localhost/45007] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-05-10 04:55:18,760 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-10 04:55:18,761 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-10 04:55:18,761 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-10 04:55:18,761 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-10 04:55:18,761 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:55:18,762 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=93c2cc329b1fe1893cfbc28c56960522, daughterA=19040b6538a6bdeed7c833cfb44614b6, daughterB=054f69c06f126aad97b290410d5d6188 2023-05-10 04:55:18,762 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:55:18,762 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:55:18,762 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-10 04:55:18,762 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6c70954228a0bbe97dfdc76861367c66, daughterA=75e2592cb2ac08bd9f617bb9d0a0c896, daughterB=b90fd344aff14f0728a80c3d09539e54 2023-05-10 04:55:18,762 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-05-10 04:55:18,762 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-10 04:55:18,763 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-10 04:55:18,763 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:55:18,763 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=4dcb62249031b560a1bc5d03e43ac855, ASSIGN 2023-05-10 04:55:18,763 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-05-10 04:55:18,763 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-05-10 04:55:18,763 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-10 04:55:18,764 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=a7ae61c05a799883e33cfd9dfad56fa1, daughterA=944df92d059326a46982ff7979f8a940, daughterB=fa1eb74e4c5521990fb9118cc6dfd407 2023-05-10 04:55:18,764 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-10 04:55:18,764 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-10 04:55:18,764 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:55:18,764 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=e0a753dfc7b1edf68920ae1543edc38c, daughterA=08557250442eebc7c86235cdf682b1a9, daughterB=e1dcbf091d2db6dbef8d701eb3d89914 2023-05-10 04:55:18,764 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:55:18,764 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:55:18,764 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-10 04:55:18,765 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-10 04:55:18,765 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,37995,1683694419499, table=testRollbackForSplitTableRegionWithReplica, region=bddd3bfc356670bc9cd9492c47d95a01 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=d2b34babccb97d3c1ac066d9e90ae654, daughterA=0141d5003cd2aac0ddef526f1a98709b, daughterB=b919ef6ce9c770ae26c6c5ee0d2ac459 2023-05-10 04:55:18,765 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-10 04:55:18,765 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-10 04:55:18,765 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-05-10 04:55:18,766 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-05-10 04:55:18,766 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-05-10 04:55:18,766 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-05-10 04:55:18,766 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ba55ae97fe6d26f2fd2a296837570536, daughterA=48042d1ae86f00fdead71e71ad8292a9, daughterB=1b26b3dca87c03702b1dc11207d7b72a 2023-05-10 04:55:18,766 DEBUG [Listener at localhost/45007] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ba55ae97fe6d26f2fd2a296837570536, daughterA=48042d1ae86f00fdead71e71ad8292a9, daughterB=1b26b3dca87c03702b1dc11207d7b72a held the lock before restarting, call acquireLock to restore it. 2023-05-10 04:55:18,767 INFO [Listener at localhost/45007] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ba55ae97fe6d26f2fd2a296837570536, daughterA=48042d1ae86f00fdead71e71ad8292a9, daughterB=1b26b3dca87c03702b1dc11207d7b72a 2023-05-10 04:55:18,767 INFO [Listener at localhost/45007] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ba55ae97fe6d26f2fd2a296837570536, daughterA=48042d1ae86f00fdead71e71ad8292a9, daughterB=1b26b3dca87c03702b1dc11207d7b72a 2023-05-10 04:55:18,767 INFO [Listener at localhost/45007] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ba55ae97fe6d26f2fd2a296837570536, daughterA=48042d1ae86f00fdead71e71ad8292a9, daughterB=1b26b3dca87c03702b1dc11207d7b72a 2023-05-10 04:55:18,767 INFO [Listener at localhost/45007] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 8 msec 2023-05-10 04:55:18,768 DEBUG [Listener at localhost/45007] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-10 04:55:18,768 DEBUG [Listener at localhost/45007] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,37995,1683694419499, table=hbase:meta, region=1588230740 2023-05-10 04:55:18,771 INFO [Listener at localhost/45007] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683694548771 2023-05-10 04:55:18,773 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ba55ae97fe6d26f2fd2a296837570536, daughterA=48042d1ae86f00fdead71e71ad8292a9, daughterB=1b26b3dca87c03702b1dc11207d7b72a 2023-05-10 04:55:18,773 DEBUG [Listener at localhost/45007] assignment.AssignmentManager(1517): Joining cluster... 2023-05-10 04:55:18,775 DEBUG [Listener at localhost/45007] assignment.RegionStateStore(147): Load hbase:meta entry region=ccfda81dc2f13ffc1429bef67d349028, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36665,1683694419452, regionLocation=jenkins-hbase4.apache.org,36665,1683694419452, openSeqNum=2 2023-05-10 04:55:18,775 DEBUG [Listener at localhost/45007] assignment.RegionStateStore(147): Load hbase:meta entry region=ba55ae97fe6d26f2fd2a296837570536, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36665,1683694419452, regionLocation=jenkins-hbase4.apache.org,36665,1683694419452, openSeqNum=2 2023-05-10 04:55:18,776 INFO [Listener at localhost/45007] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-10 04:55:18,776 INFO [Listener at localhost/45007] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683694578776 2023-05-10 04:55:18,776 INFO [Listener at localhost/45007] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683694638776 2023-05-10 04:55:18,776 INFO [Listener at localhost/45007] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-05-10 04:55:18,776 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-10 04:55:18,778 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=ba55ae97fe6d26f2fd2a296837570536, UNASSIGN}] 2023-05-10 04:55:18,778 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-10 04:55:18,778 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ba55ae97fe6d26f2fd2a296837570536, daughterA=48042d1ae86f00fdead71e71ad8292a9, daughterB=1b26b3dca87c03702b1dc11207d7b72a 2023-05-10 04:55:18,778 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-10 04:55:18,778 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ba55ae97fe6d26f2fd2a296837570536, daughterA=48042d1ae86f00fdead71e71ad8292a9, daughterB=1b26b3dca87c03702b1dc11207d7b72a at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-10 04:55:19,026 INFO [Listener at localhost/45007] procedure.MasterProcedureTestingUtility(454): Restart 2 exec state=pid=112, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ba55ae97fe6d26f2fd2a296837570536, daughterA=48042d1ae86f00fdead71e71ad8292a9, daughterB=1b26b3dca87c03702b1dc11207d7b72a 2023-05-10 04:55:19,026 INFO [Listener at localhost/45007] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-10 04:55:19,026 INFO [Listener at localhost/45007] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-10 04:55:19,026 INFO [Listener at localhost/45007] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-10 04:55:19,027 ERROR [Listener at localhost/45007] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-10 04:55:19,027 INFO [Listener at localhost/45007] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-10 04:55:19,027 INFO [Listener at localhost/45007] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-10 04:55:19,027 INFO [Listener at localhost/45007] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-10 04:55:19,027 INFO [Listener at localhost/45007] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-10 04:55:19,028 INFO [Listener at localhost/45007] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-10 04:55:19,029 DEBUG [Listener at localhost/45007] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-05-10 04:55:19,029 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-10 04:55:19,030 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-10 04:55:19,030 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-10 04:55:19,030 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-10 04:55:19,030 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:55:19,030 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=93c2cc329b1fe1893cfbc28c56960522, daughterA=19040b6538a6bdeed7c833cfb44614b6, daughterB=054f69c06f126aad97b290410d5d6188 2023-05-10 04:55:19,030 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:55:19,031 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:55:19,031 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-10 04:55:19,031 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6c70954228a0bbe97dfdc76861367c66, daughterA=75e2592cb2ac08bd9f617bb9d0a0c896, daughterB=b90fd344aff14f0728a80c3d09539e54 2023-05-10 04:55:19,031 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-05-10 04:55:19,031 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-10 04:55:19,031 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-10 04:55:19,032 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:55:19,032 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=4dcb62249031b560a1bc5d03e43ac855, ASSIGN 2023-05-10 04:55:19,032 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-05-10 04:55:19,032 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-05-10 04:55:19,032 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-10 04:55:19,032 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=a7ae61c05a799883e33cfd9dfad56fa1, daughterA=944df92d059326a46982ff7979f8a940, daughterB=fa1eb74e4c5521990fb9118cc6dfd407 2023-05-10 04:55:19,032 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-10 04:55:19,032 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-10 04:55:19,033 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:55:19,033 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=e0a753dfc7b1edf68920ae1543edc38c, daughterA=08557250442eebc7c86235cdf682b1a9, daughterB=e1dcbf091d2db6dbef8d701eb3d89914 2023-05-10 04:55:19,033 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:55:19,033 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:55:19,034 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-10 04:55:19,034 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-10 04:55:19,034 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,37995,1683694419499, table=testRollbackForSplitTableRegionWithReplica, region=bddd3bfc356670bc9cd9492c47d95a01 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=d2b34babccb97d3c1ac066d9e90ae654, daughterA=0141d5003cd2aac0ddef526f1a98709b, daughterB=b919ef6ce9c770ae26c6c5ee0d2ac459 2023-05-10 04:55:19,034 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-10 04:55:19,035 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-10 04:55:19,035 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-05-10 04:55:19,035 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-05-10 04:55:19,035 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-05-10 04:55:19,036 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-05-10 04:55:19,036 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ba55ae97fe6d26f2fd2a296837570536, daughterA=48042d1ae86f00fdead71e71ad8292a9, daughterB=1b26b3dca87c03702b1dc11207d7b72a 2023-05-10 04:55:19,036 DEBUG [Listener at localhost/45007] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ba55ae97fe6d26f2fd2a296837570536, daughterA=48042d1ae86f00fdead71e71ad8292a9, daughterB=1b26b3dca87c03702b1dc11207d7b72a held the lock before restarting, call acquireLock to restore it. 2023-05-10 04:55:19,036 INFO [Listener at localhost/45007] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ba55ae97fe6d26f2fd2a296837570536, daughterA=48042d1ae86f00fdead71e71ad8292a9, daughterB=1b26b3dca87c03702b1dc11207d7b72a 2023-05-10 04:55:19,036 INFO [Listener at localhost/45007] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ba55ae97fe6d26f2fd2a296837570536, daughterA=48042d1ae86f00fdead71e71ad8292a9, daughterB=1b26b3dca87c03702b1dc11207d7b72a 2023-05-10 04:55:19,036 INFO [Listener at localhost/45007] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ba55ae97fe6d26f2fd2a296837570536, daughterA=48042d1ae86f00fdead71e71ad8292a9, daughterB=1b26b3dca87c03702b1dc11207d7b72a 2023-05-10 04:55:19,036 INFO [Listener at localhost/45007] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 8 msec 2023-05-10 04:55:19,037 DEBUG [Listener at localhost/45007] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-10 04:55:19,038 DEBUG [Listener at localhost/45007] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,37995,1683694419499, table=hbase:meta, region=1588230740 2023-05-10 04:55:19,038 INFO [Listener at localhost/45007] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683694549038 2023-05-10 04:55:19,038 DEBUG [Listener at localhost/45007] assignment.AssignmentManager(1517): Joining cluster... 2023-05-10 04:55:19,039 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ba55ae97fe6d26f2fd2a296837570536, daughterA=48042d1ae86f00fdead71e71ad8292a9, daughterB=1b26b3dca87c03702b1dc11207d7b72a 2023-05-10 04:55:19,042 DEBUG [Listener at localhost/45007] assignment.RegionStateStore(147): Load hbase:meta entry region=ccfda81dc2f13ffc1429bef67d349028, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36665,1683694419452, regionLocation=jenkins-hbase4.apache.org,36665,1683694419452, openSeqNum=2 2023-05-10 04:55:19,042 DEBUG [Listener at localhost/45007] assignment.RegionStateStore(147): Load hbase:meta entry region=ba55ae97fe6d26f2fd2a296837570536, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36665,1683694419452, regionLocation=jenkins-hbase4.apache.org,36665,1683694419452, openSeqNum=2 2023-05-10 04:55:19,042 INFO [Listener at localhost/45007] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-10 04:55:19,042 INFO [Listener at localhost/45007] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683694579042 2023-05-10 04:55:19,043 INFO [Listener at localhost/45007] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683694639043 2023-05-10 04:55:19,043 INFO [Listener at localhost/45007] assignment.AssignmentManager(1536): Joined the cluster in 4 msec 2023-05-10 04:55:19,043 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=ba55ae97fe6d26f2fd2a296837570536, UNASSIGN}] 2023-05-10 04:55:19,043 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-10 04:55:19,045 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=ba55ae97fe6d26f2fd2a296837570536, UNASSIGN 2023-05-10 04:55:19,045 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=ba55ae97fe6d26f2fd2a296837570536, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:19,045 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1683694516443.ba55ae97fe6d26f2fd2a296837570536.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1683694519045"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683694519045"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683694519045"}]},"ts":"1683694519045"} 2023-05-10 04:55:19,046 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=114, ppid=113, state=RUNNABLE; CloseRegionProcedure ba55ae97fe6d26f2fd2a296837570536, server=jenkins-hbase4.apache.org,36665,1683694419452}] 2023-05-10 04:55:19,113 ERROR [BP-726857689-172.31.14.131-1683694414559 heartbeating to localhost/127.0.0.1:38753] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-726857689-172.31.14.131-1683694414559 (Datanode Uuid 0bc2fe53-c3db-437e-a8da-2e5e8eeaaf20) service to localhost/127.0.0.1:38753 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-05-10 04:55:19,198 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:19,198 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing ba55ae97fe6d26f2fd2a296837570536, disabling compactions & flushes 2023-05-10 04:55:19,198 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testRecoveryAndDoubleExecution,,1683694516443.ba55ae97fe6d26f2fd2a296837570536. 2023-05-10 04:55:19,198 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,,1683694516443.ba55ae97fe6d26f2fd2a296837570536. 2023-05-10 04:55:19,199 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testRecoveryAndDoubleExecution,,1683694516443.ba55ae97fe6d26f2fd2a296837570536. after waiting 0 ms 2023-05-10 04:55:19,199 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testRecoveryAndDoubleExecution,,1683694516443.ba55ae97fe6d26f2fd2a296837570536. 2023-05-10 04:55:19,199 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing ba55ae97fe6d26f2fd2a296837570536 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-05-10 04:55:19,207 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/.tmp/cf1/4e150d9797e44e02a343bd11ff7b3dc3 2023-05-10 04:55:19,222 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/.tmp/cf2/34f3b8d7d8774320929c1c1bbcc1f132 2023-05-10 04:55:19,225 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/.tmp/cf1/4e150d9797e44e02a343bd11ff7b3dc3 as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf1/4e150d9797e44e02a343bd11ff7b3dc3 2023-05-10 04:55:19,229 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf1/4e150d9797e44e02a343bd11ff7b3dc3, entries=8, sequenceid=82, filesize=5.0 K 2023-05-10 04:55:19,229 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/.tmp/cf2/34f3b8d7d8774320929c1c1bbcc1f132 as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf2/34f3b8d7d8774320929c1c1bbcc1f132 2023-05-10 04:55:19,233 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf2/34f3b8d7d8774320929c1c1bbcc1f132, entries=8, sequenceid=82, filesize=5.0 K 2023-05-10 04:55:19,233 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for ba55ae97fe6d26f2fd2a296837570536 in 34ms, sequenceid=82, compaction requested=true 2023-05-10 04:55:19,236 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-05-10 04:55:19,237 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-10 04:55:19,238 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testRecoveryAndDoubleExecution,,1683694516443.ba55ae97fe6d26f2fd2a296837570536. 2023-05-10 04:55:19,238 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for ba55ae97fe6d26f2fd2a296837570536: 2023-05-10 04:55:19,240 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:19,240 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=ba55ae97fe6d26f2fd2a296837570536, regionState=CLOSED 2023-05-10 04:55:19,240 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,,1683694516443.ba55ae97fe6d26f2fd2a296837570536.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1683694519240"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683694519240"}]},"ts":"1683694519240"} 2023-05-10 04:55:19,243 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=114, resume processing ppid=113 2023-05-10 04:55:19,243 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure ba55ae97fe6d26f2fd2a296837570536, server=jenkins-hbase4.apache.org,36665,1683694419452 in 195 msec 2023-05-10 04:55:19,244 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=113, resume processing ppid=112 2023-05-10 04:55:19,244 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=ba55ae97fe6d26f2fd2a296837570536, UNASSIGN in 200 msec 2023-05-10 04:55:19,246 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-10 04:55:19,246 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ba55ae97fe6d26f2fd2a296837570536, daughterA=48042d1ae86f00fdead71e71ad8292a9, daughterB=1b26b3dca87c03702b1dc11207d7b72a 2023-05-10 04:55:19,246 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-10 04:55:19,246 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ba55ae97fe6d26f2fd2a296837570536, daughterA=48042d1ae86f00fdead71e71ad8292a9, daughterB=1b26b3dca87c03702b1dc11207d7b72a at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-10 04:55:19,293 INFO [Listener at localhost/45007] procedure.MasterProcedureTestingUtility(454): Restart 3 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ba55ae97fe6d26f2fd2a296837570536, daughterA=48042d1ae86f00fdead71e71ad8292a9, daughterB=1b26b3dca87c03702b1dc11207d7b72a 2023-05-10 04:55:19,293 INFO [Listener at localhost/45007] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-10 04:55:19,293 INFO [Listener at localhost/45007] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-10 04:55:19,293 INFO [Listener at localhost/45007] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-10 04:55:19,293 ERROR [Listener at localhost/45007] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-10 04:55:19,293 INFO [Listener at localhost/45007] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-10 04:55:19,293 INFO [Listener at localhost/45007] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-10 04:55:19,293 INFO [Listener at localhost/45007] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-10 04:55:19,294 INFO [Listener at localhost/45007] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-10 04:55:19,294 INFO [Listener at localhost/45007] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-10 04:55:19,295 DEBUG [Listener at localhost/45007] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-05-10 04:55:19,295 DEBUG [Listener at localhost/45007] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-05-10 04:55:19,295 DEBUG [Listener at localhost/45007] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-05-10 04:55:19,296 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-10 04:55:19,296 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-10 04:55:19,296 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-10 04:55:19,296 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-10 04:55:19,296 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:55:19,297 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=93c2cc329b1fe1893cfbc28c56960522, daughterA=19040b6538a6bdeed7c833cfb44614b6, daughterB=054f69c06f126aad97b290410d5d6188 2023-05-10 04:55:19,297 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:55:19,297 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:55:19,297 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-10 04:55:19,297 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6c70954228a0bbe97dfdc76861367c66, daughterA=75e2592cb2ac08bd9f617bb9d0a0c896, daughterB=b90fd344aff14f0728a80c3d09539e54 2023-05-10 04:55:19,297 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-05-10 04:55:19,297 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-10 04:55:19,297 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-10 04:55:19,298 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:55:19,298 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=4dcb62249031b560a1bc5d03e43ac855, ASSIGN 2023-05-10 04:55:19,298 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-05-10 04:55:19,298 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-05-10 04:55:19,298 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-10 04:55:19,299 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=a7ae61c05a799883e33cfd9dfad56fa1, daughterA=944df92d059326a46982ff7979f8a940, daughterB=fa1eb74e4c5521990fb9118cc6dfd407 2023-05-10 04:55:19,299 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-10 04:55:19,299 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-10 04:55:19,299 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:55:19,299 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=e0a753dfc7b1edf68920ae1543edc38c, daughterA=08557250442eebc7c86235cdf682b1a9, daughterB=e1dcbf091d2db6dbef8d701eb3d89914 2023-05-10 04:55:19,299 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:55:19,299 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:55:19,299 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-10 04:55:19,300 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-10 04:55:19,300 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,37995,1683694419499, table=testRollbackForSplitTableRegionWithReplica, region=bddd3bfc356670bc9cd9492c47d95a01 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=d2b34babccb97d3c1ac066d9e90ae654, daughterA=0141d5003cd2aac0ddef526f1a98709b, daughterB=b919ef6ce9c770ae26c6c5ee0d2ac459 2023-05-10 04:55:19,300 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-10 04:55:19,300 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-10 04:55:19,300 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-05-10 04:55:19,300 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-05-10 04:55:19,300 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-05-10 04:55:19,301 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-05-10 04:55:19,301 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ba55ae97fe6d26f2fd2a296837570536, daughterA=48042d1ae86f00fdead71e71ad8292a9, daughterB=1b26b3dca87c03702b1dc11207d7b72a 2023-05-10 04:55:19,301 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=ba55ae97fe6d26f2fd2a296837570536, UNASSIGN 2023-05-10 04:55:19,301 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure ba55ae97fe6d26f2fd2a296837570536, server=jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:19,301 DEBUG [Listener at localhost/45007] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ba55ae97fe6d26f2fd2a296837570536, daughterA=48042d1ae86f00fdead71e71ad8292a9, daughterB=1b26b3dca87c03702b1dc11207d7b72a held the lock before restarting, call acquireLock to restore it. 2023-05-10 04:55:19,301 INFO [Listener at localhost/45007] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ba55ae97fe6d26f2fd2a296837570536, daughterA=48042d1ae86f00fdead71e71ad8292a9, daughterB=1b26b3dca87c03702b1dc11207d7b72a 2023-05-10 04:55:19,301 INFO [Listener at localhost/45007] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ba55ae97fe6d26f2fd2a296837570536, daughterA=48042d1ae86f00fdead71e71ad8292a9, daughterB=1b26b3dca87c03702b1dc11207d7b72a 2023-05-10 04:55:19,301 INFO [Listener at localhost/45007] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ba55ae97fe6d26f2fd2a296837570536, daughterA=48042d1ae86f00fdead71e71ad8292a9, daughterB=1b26b3dca87c03702b1dc11207d7b72a 2023-05-10 04:55:19,301 DEBUG [Listener at localhost/45007] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=ba55ae97fe6d26f2fd2a296837570536, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-05-10 04:55:19,301 DEBUG [Listener at localhost/45007] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure ba55ae97fe6d26f2fd2a296837570536, server=jenkins-hbase4.apache.org,36665,1683694419452 didn't hold the lock before restarting, skip acquiring lock. 2023-05-10 04:55:19,301 INFO [Listener at localhost/45007] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-05-10 04:55:19,302 DEBUG [Listener at localhost/45007] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-10 04:55:19,303 DEBUG [Listener at localhost/45007] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,37995,1683694419499, table=hbase:meta, region=1588230740 2023-05-10 04:55:19,303 INFO [Listener at localhost/45007] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683694549303 2023-05-10 04:55:19,304 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ba55ae97fe6d26f2fd2a296837570536, daughterA=48042d1ae86f00fdead71e71ad8292a9, daughterB=1b26b3dca87c03702b1dc11207d7b72a 2023-05-10 04:55:19,304 DEBUG [Listener at localhost/45007] assignment.AssignmentManager(1517): Joining cluster... 2023-05-10 04:55:19,307 DEBUG [Listener at localhost/45007] assignment.RegionStateStore(147): Load hbase:meta entry region=ccfda81dc2f13ffc1429bef67d349028, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36665,1683694419452, regionLocation=jenkins-hbase4.apache.org,36665,1683694419452, openSeqNum=2 2023-05-10 04:55:19,308 DEBUG [Listener at localhost/45007] assignment.RegionStateStore(147): Load hbase:meta entry region=ba55ae97fe6d26f2fd2a296837570536, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,36665,1683694419452, regionLocation=jenkins-hbase4.apache.org,36665,1683694419452, openSeqNum=2 2023-05-10 04:55:19,308 INFO [Listener at localhost/45007] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-10 04:55:19,308 INFO [Listener at localhost/45007] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683694579308 2023-05-10 04:55:19,308 INFO [Listener at localhost/45007] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683694639308 2023-05-10 04:55:19,308 INFO [Listener at localhost/45007] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-05-10 04:55:19,310 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-10 04:55:19,319 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=112 splitting 14 storefiles, region=ba55ae97fe6d26f2fd2a296837570536, threads=14 2023-05-10 04:55:19,319 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf2/14e73a6c8cd44ed2ab40f783dad882ca for region: ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:19,319 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf2/1747520752eb459e951b7491aa7c62a3 for region: ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:19,319 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf2/34f3b8d7d8774320929c1c1bbcc1f132 for region: ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:19,319 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf2/c332f2f6f5a44943a5e8c97725d45512 for region: ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:19,320 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf2/c4912c4098564070a195e3fd918ecec2 for region: ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:19,320 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf2/c845912c71644277a0aed468d0621d59 for region: ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:19,320 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf2/eb76ae13368e4a23855b5680af181f32 for region: ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:19,320 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf1/17ee8f61ba9d4f678887f8a52b767dbb for region: ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:19,320 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf1/3488050453f04078bc1365497a2e3f88 for region: ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:19,320 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf1/4e150d9797e44e02a343bd11ff7b3dc3 for region: ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:19,320 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf1/aa810537cf12426fa1bcdd14225b68ee for region: ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:19,320 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf1/b3e182af00c84ddb81eb7331183b802e for region: ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:19,321 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf1/bd5a17faa7564deb857bb9ec28719ee6 for region: ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:19,321 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf1/c2d599f271fb497b8b272a3bfd391f6e for region: ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:19,447 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf2/14e73a6c8cd44ed2ab40f783dad882ca for region: ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:19,492 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf2/c332f2f6f5a44943a5e8c97725d45512 for region: ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:19,510 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf1/b3e182af00c84ddb81eb7331183b802e for region: ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:19,512 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf1/aa810537cf12426fa1bcdd14225b68ee for region: ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:19,539 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf2/c845912c71644277a0aed468d0621d59 for region: ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:19,547 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf1/3488050453f04078bc1365497a2e3f88 for region: ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:19,551 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf2/34f3b8d7d8774320929c1c1bbcc1f132 for region: ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:19,559 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf1/17ee8f61ba9d4f678887f8a52b767dbb for region: ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:19,561 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf1/c2d599f271fb497b8b272a3bfd391f6e for region: ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:19,563 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf2/1747520752eb459e951b7491aa7c62a3 for region: ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:19,563 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf2/c4912c4098564070a195e3fd918ecec2 for region: ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:19,563 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf2/eb76ae13368e4a23855b5680af181f32 for region: ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:19,564 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf1/bd5a17faa7564deb857bb9ec28719ee6 for region: ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:19,564 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf1/4e150d9797e44e02a343bd11ff7b3dc3 for region: ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:19,564 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=112 split storefiles for region ba55ae97fe6d26f2fd2a296837570536 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-05-10 04:55:19,601 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-10 04:55:19,601 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ba55ae97fe6d26f2fd2a296837570536, daughterA=48042d1ae86f00fdead71e71ad8292a9, daughterB=1b26b3dca87c03702b1dc11207d7b72a 2023-05-10 04:55:19,601 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-10 04:55:19,601 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ba55ae97fe6d26f2fd2a296837570536, daughterA=48042d1ae86f00fdead71e71ad8292a9, daughterB=1b26b3dca87c03702b1dc11207d7b72a at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-10 04:55:19,656 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-05-10 04:55:19,808 INFO [Listener at localhost/45007] procedure.MasterProcedureTestingUtility(454): Restart 4 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ba55ae97fe6d26f2fd2a296837570536, daughterA=48042d1ae86f00fdead71e71ad8292a9, daughterB=1b26b3dca87c03702b1dc11207d7b72a 2023-05-10 04:55:19,808 INFO [Listener at localhost/45007] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-10 04:55:19,808 INFO [Listener at localhost/45007] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-10 04:55:19,809 INFO [Listener at localhost/45007] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-10 04:55:19,809 ERROR [Listener at localhost/45007] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-10 04:55:19,809 INFO [Listener at localhost/45007] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-10 04:55:19,809 INFO [Listener at localhost/45007] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-10 04:55:19,809 INFO [Listener at localhost/45007] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-10 04:55:19,809 INFO [Listener at localhost/45007] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-10 04:55:19,810 INFO [Listener at localhost/45007] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-10 04:55:19,811 DEBUG [Listener at localhost/45007] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-05-10 04:55:19,811 DEBUG [Listener at localhost/45007] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-05-10 04:55:19,811 DEBUG [Listener at localhost/45007] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-05-10 04:55:19,812 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-10 04:55:19,812 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-10 04:55:19,812 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-10 04:55:19,812 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-10 04:55:19,813 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:55:19,813 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=93c2cc329b1fe1893cfbc28c56960522, daughterA=19040b6538a6bdeed7c833cfb44614b6, daughterB=054f69c06f126aad97b290410d5d6188 2023-05-10 04:55:19,813 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:55:19,813 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:55:19,813 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-10 04:55:19,814 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6c70954228a0bbe97dfdc76861367c66, daughterA=75e2592cb2ac08bd9f617bb9d0a0c896, daughterB=b90fd344aff14f0728a80c3d09539e54 2023-05-10 04:55:19,814 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-05-10 04:55:19,814 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-10 04:55:19,814 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-10 04:55:19,814 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:55:19,814 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=4dcb62249031b560a1bc5d03e43ac855, ASSIGN 2023-05-10 04:55:19,815 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-05-10 04:55:19,815 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-05-10 04:55:19,815 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-10 04:55:19,815 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=a7ae61c05a799883e33cfd9dfad56fa1, daughterA=944df92d059326a46982ff7979f8a940, daughterB=fa1eb74e4c5521990fb9118cc6dfd407 2023-05-10 04:55:19,815 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-10 04:55:19,815 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-10 04:55:19,815 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:55:19,816 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=e0a753dfc7b1edf68920ae1543edc38c, daughterA=08557250442eebc7c86235cdf682b1a9, daughterB=e1dcbf091d2db6dbef8d701eb3d89914 2023-05-10 04:55:19,816 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:55:19,816 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:55:19,816 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-10 04:55:19,816 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-10 04:55:19,816 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,37995,1683694419499, table=testRollbackForSplitTableRegionWithReplica, region=bddd3bfc356670bc9cd9492c47d95a01 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=d2b34babccb97d3c1ac066d9e90ae654, daughterA=0141d5003cd2aac0ddef526f1a98709b, daughterB=b919ef6ce9c770ae26c6c5ee0d2ac459 2023-05-10 04:55:19,817 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-10 04:55:19,817 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-10 04:55:19,817 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-05-10 04:55:19,817 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-05-10 04:55:19,817 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-05-10 04:55:19,817 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-05-10 04:55:19,817 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ba55ae97fe6d26f2fd2a296837570536, daughterA=48042d1ae86f00fdead71e71ad8292a9, daughterB=1b26b3dca87c03702b1dc11207d7b72a 2023-05-10 04:55:19,817 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=ba55ae97fe6d26f2fd2a296837570536, UNASSIGN 2023-05-10 04:55:19,818 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure ba55ae97fe6d26f2fd2a296837570536, server=jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:19,818 DEBUG [Listener at localhost/45007] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ba55ae97fe6d26f2fd2a296837570536, daughterA=48042d1ae86f00fdead71e71ad8292a9, daughterB=1b26b3dca87c03702b1dc11207d7b72a held the lock before restarting, call acquireLock to restore it. 2023-05-10 04:55:19,818 INFO [Listener at localhost/45007] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ba55ae97fe6d26f2fd2a296837570536, daughterA=48042d1ae86f00fdead71e71ad8292a9, daughterB=1b26b3dca87c03702b1dc11207d7b72a 2023-05-10 04:55:19,818 INFO [Listener at localhost/45007] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ba55ae97fe6d26f2fd2a296837570536, daughterA=48042d1ae86f00fdead71e71ad8292a9, daughterB=1b26b3dca87c03702b1dc11207d7b72a 2023-05-10 04:55:19,818 INFO [Listener at localhost/45007] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ba55ae97fe6d26f2fd2a296837570536, daughterA=48042d1ae86f00fdead71e71ad8292a9, daughterB=1b26b3dca87c03702b1dc11207d7b72a 2023-05-10 04:55:19,818 DEBUG [Listener at localhost/45007] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=ba55ae97fe6d26f2fd2a296837570536, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-05-10 04:55:19,818 DEBUG [Listener at localhost/45007] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure ba55ae97fe6d26f2fd2a296837570536, server=jenkins-hbase4.apache.org,36665,1683694419452 didn't hold the lock before restarting, skip acquiring lock. 2023-05-10 04:55:19,818 INFO [Listener at localhost/45007] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 8 msec 2023-05-10 04:55:19,819 DEBUG [Listener at localhost/45007] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-10 04:55:19,819 DEBUG [Listener at localhost/45007] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,37995,1683694419499, table=hbase:meta, region=1588230740 2023-05-10 04:55:19,820 INFO [Listener at localhost/45007] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683694549820 2023-05-10 04:55:19,821 DEBUG [Listener at localhost/45007] assignment.AssignmentManager(1517): Joining cluster... 2023-05-10 04:55:19,821 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ba55ae97fe6d26f2fd2a296837570536, daughterA=48042d1ae86f00fdead71e71ad8292a9, daughterB=1b26b3dca87c03702b1dc11207d7b72a 2023-05-10 04:55:19,823 DEBUG [Listener at localhost/45007] assignment.RegionStateStore(147): Load hbase:meta entry region=ccfda81dc2f13ffc1429bef67d349028, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36665,1683694419452, regionLocation=jenkins-hbase4.apache.org,36665,1683694419452, openSeqNum=2 2023-05-10 04:55:19,823 DEBUG [Listener at localhost/45007] assignment.RegionStateStore(147): Load hbase:meta entry region=ba55ae97fe6d26f2fd2a296837570536, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,36665,1683694419452, regionLocation=jenkins-hbase4.apache.org,36665,1683694419452, openSeqNum=2 2023-05-10 04:55:19,824 INFO [Listener at localhost/45007] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-10 04:55:19,824 INFO [Listener at localhost/45007] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683694579824 2023-05-10 04:55:19,824 INFO [Listener at localhost/45007] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683694639824 2023-05-10 04:55:19,824 INFO [Listener at localhost/45007] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-05-10 04:55:19,831 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=112 splitting 14 storefiles, region=ba55ae97fe6d26f2fd2a296837570536, threads=14 2023-05-10 04:55:19,832 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf2/14e73a6c8cd44ed2ab40f783dad882ca for region: ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:19,832 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf2/34f3b8d7d8774320929c1c1bbcc1f132 for region: ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:19,832 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf2/1747520752eb459e951b7491aa7c62a3 for region: ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:19,832 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf2/c332f2f6f5a44943a5e8c97725d45512 for region: ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:19,832 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf2/c845912c71644277a0aed468d0621d59 for region: ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:19,832 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf2/c4912c4098564070a195e3fd918ecec2 for region: ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:19,832 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf2/eb76ae13368e4a23855b5680af181f32 for region: ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:19,833 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf1/17ee8f61ba9d4f678887f8a52b767dbb for region: ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:19,833 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf1/4e150d9797e44e02a343bd11ff7b3dc3 for region: ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:19,833 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf1/3488050453f04078bc1365497a2e3f88 for region: ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:19,833 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf1/aa810537cf12426fa1bcdd14225b68ee for region: ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:19,833 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf1/b3e182af00c84ddb81eb7331183b802e for region: ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:19,834 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf1/bd5a17faa7564deb857bb9ec28719ee6 for region: ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:19,836 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf1/c2d599f271fb497b8b272a3bfd391f6e for region: ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:19,967 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf2/c4912c4098564070a195e3fd918ecec2 for region: ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:19,975 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf2/14e73a6c8cd44ed2ab40f783dad882ca for region: ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:19,983 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf2/eb76ae13368e4a23855b5680af181f32 for region: ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:19,996 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf2/c332f2f6f5a44943a5e8c97725d45512 for region: ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:19,997 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf2/1747520752eb459e951b7491aa7c62a3 for region: ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:20,011 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf1/17ee8f61ba9d4f678887f8a52b767dbb for region: ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:20,016 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf1/c2d599f271fb497b8b272a3bfd391f6e for region: ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:20,022 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf1/b3e182af00c84ddb81eb7331183b802e for region: ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:20,024 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf2/c845912c71644277a0aed468d0621d59 for region: ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:20,026 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf1/bd5a17faa7564deb857bb9ec28719ee6 for region: ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:20,027 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf1/4e150d9797e44e02a343bd11ff7b3dc3 for region: ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:20,027 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf1/3488050453f04078bc1365497a2e3f88 for region: ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:20,029 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf1/aa810537cf12426fa1bcdd14225b68ee for region: ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:20,029 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf2/34f3b8d7d8774320929c1c1bbcc1f132 for region: ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:20,029 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=112 split storefiles for region ba55ae97fe6d26f2fd2a296837570536 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-05-10 04:55:20,056 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-10 04:55:20,061 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-05-10 04:55:20,062 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-05-10 04:55:20,062 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-10 04:55:20,062 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ba55ae97fe6d26f2fd2a296837570536, daughterA=48042d1ae86f00fdead71e71ad8292a9, daughterB=1b26b3dca87c03702b1dc11207d7b72a 2023-05-10 04:55:20,062 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-10 04:55:20,062 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ba55ae97fe6d26f2fd2a296837570536, daughterA=48042d1ae86f00fdead71e71ad8292a9, daughterB=1b26b3dca87c03702b1dc11207d7b72a at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-10 04:55:20,074 INFO [Listener at localhost/45007] procedure.MasterProcedureTestingUtility(454): Restart 5 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ba55ae97fe6d26f2fd2a296837570536, daughterA=48042d1ae86f00fdead71e71ad8292a9, daughterB=1b26b3dca87c03702b1dc11207d7b72a 2023-05-10 04:55:20,074 INFO [Listener at localhost/45007] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-10 04:55:20,074 INFO [Listener at localhost/45007] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-10 04:55:20,074 INFO [Listener at localhost/45007] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-10 04:55:20,074 ERROR [Listener at localhost/45007] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-10 04:55:20,074 INFO [Listener at localhost/45007] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-10 04:55:20,074 INFO [Listener at localhost/45007] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-10 04:55:20,075 INFO [Listener at localhost/45007] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-10 04:55:20,075 INFO [Listener at localhost/45007] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-10 04:55:20,075 INFO [Listener at localhost/45007] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-10 04:55:20,076 DEBUG [Listener at localhost/45007] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-05-10 04:55:20,076 DEBUG [Listener at localhost/45007] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-05-10 04:55:20,076 DEBUG [Listener at localhost/45007] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-05-10 04:55:20,076 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-10 04:55:20,077 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-10 04:55:20,077 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-10 04:55:20,077 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-10 04:55:20,077 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:55:20,077 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=93c2cc329b1fe1893cfbc28c56960522, daughterA=19040b6538a6bdeed7c833cfb44614b6, daughterB=054f69c06f126aad97b290410d5d6188 2023-05-10 04:55:20,077 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:55:20,078 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:55:20,078 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-10 04:55:20,078 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6c70954228a0bbe97dfdc76861367c66, daughterA=75e2592cb2ac08bd9f617bb9d0a0c896, daughterB=b90fd344aff14f0728a80c3d09539e54 2023-05-10 04:55:20,078 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-05-10 04:55:20,078 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-10 04:55:20,078 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-10 04:55:20,078 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:55:20,078 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=4dcb62249031b560a1bc5d03e43ac855, ASSIGN 2023-05-10 04:55:20,079 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-05-10 04:55:20,079 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-05-10 04:55:20,079 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-10 04:55:20,079 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=a7ae61c05a799883e33cfd9dfad56fa1, daughterA=944df92d059326a46982ff7979f8a940, daughterB=fa1eb74e4c5521990fb9118cc6dfd407 2023-05-10 04:55:20,079 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-10 04:55:20,079 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-10 04:55:20,079 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:55:20,079 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=e0a753dfc7b1edf68920ae1543edc38c, daughterA=08557250442eebc7c86235cdf682b1a9, daughterB=e1dcbf091d2db6dbef8d701eb3d89914 2023-05-10 04:55:20,080 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:55:20,080 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:55:20,080 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-10 04:55:20,080 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-10 04:55:20,080 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,37995,1683694419499, table=testRollbackForSplitTableRegionWithReplica, region=bddd3bfc356670bc9cd9492c47d95a01 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=d2b34babccb97d3c1ac066d9e90ae654, daughterA=0141d5003cd2aac0ddef526f1a98709b, daughterB=b919ef6ce9c770ae26c6c5ee0d2ac459 2023-05-10 04:55:20,080 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-10 04:55:20,080 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-10 04:55:20,081 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-05-10 04:55:20,081 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-05-10 04:55:20,081 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-05-10 04:55:20,081 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-05-10 04:55:20,081 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ba55ae97fe6d26f2fd2a296837570536, daughterA=48042d1ae86f00fdead71e71ad8292a9, daughterB=1b26b3dca87c03702b1dc11207d7b72a 2023-05-10 04:55:20,081 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=ba55ae97fe6d26f2fd2a296837570536, UNASSIGN 2023-05-10 04:55:20,081 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure ba55ae97fe6d26f2fd2a296837570536, server=jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:20,081 DEBUG [Listener at localhost/45007] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ba55ae97fe6d26f2fd2a296837570536, daughterA=48042d1ae86f00fdead71e71ad8292a9, daughterB=1b26b3dca87c03702b1dc11207d7b72a held the lock before restarting, call acquireLock to restore it. 2023-05-10 04:55:20,081 INFO [Listener at localhost/45007] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ba55ae97fe6d26f2fd2a296837570536, daughterA=48042d1ae86f00fdead71e71ad8292a9, daughterB=1b26b3dca87c03702b1dc11207d7b72a 2023-05-10 04:55:20,082 INFO [Listener at localhost/45007] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ba55ae97fe6d26f2fd2a296837570536, daughterA=48042d1ae86f00fdead71e71ad8292a9, daughterB=1b26b3dca87c03702b1dc11207d7b72a 2023-05-10 04:55:20,082 INFO [Listener at localhost/45007] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ba55ae97fe6d26f2fd2a296837570536, daughterA=48042d1ae86f00fdead71e71ad8292a9, daughterB=1b26b3dca87c03702b1dc11207d7b72a 2023-05-10 04:55:20,082 DEBUG [Listener at localhost/45007] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=ba55ae97fe6d26f2fd2a296837570536, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-05-10 04:55:20,082 DEBUG [Listener at localhost/45007] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure ba55ae97fe6d26f2fd2a296837570536, server=jenkins-hbase4.apache.org,36665,1683694419452 didn't hold the lock before restarting, skip acquiring lock. 2023-05-10 04:55:20,082 INFO [Listener at localhost/45007] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-05-10 04:55:20,082 DEBUG [Listener at localhost/45007] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-10 04:55:20,083 DEBUG [Listener at localhost/45007] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,37995,1683694419499, table=hbase:meta, region=1588230740 2023-05-10 04:55:20,083 INFO [Listener at localhost/45007] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683694550083 2023-05-10 04:55:20,084 DEBUG [Listener at localhost/45007] assignment.AssignmentManager(1517): Joining cluster... 2023-05-10 04:55:20,084 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ba55ae97fe6d26f2fd2a296837570536, daughterA=48042d1ae86f00fdead71e71ad8292a9, daughterB=1b26b3dca87c03702b1dc11207d7b72a 2023-05-10 04:55:20,087 DEBUG [Listener at localhost/45007] assignment.RegionStateStore(147): Load hbase:meta entry region=ccfda81dc2f13ffc1429bef67d349028, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36665,1683694419452, regionLocation=jenkins-hbase4.apache.org,36665,1683694419452, openSeqNum=2 2023-05-10 04:55:20,087 DEBUG [Listener at localhost/45007] assignment.RegionStateStore(147): Load hbase:meta entry region=ba55ae97fe6d26f2fd2a296837570536, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,36665,1683694419452, regionLocation=jenkins-hbase4.apache.org,36665,1683694419452, openSeqNum=2 2023-05-10 04:55:20,087 INFO [Listener at localhost/45007] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-10 04:55:20,087 INFO [Listener at localhost/45007] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683694580087 2023-05-10 04:55:20,087 INFO [Listener at localhost/45007] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683694640087 2023-05-10 04:55:20,087 INFO [Listener at localhost/45007] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-05-10 04:55:20,090 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-10 04:55:20,091 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-10 04:55:20,092 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ba55ae97fe6d26f2fd2a296837570536, daughterA=48042d1ae86f00fdead71e71ad8292a9, daughterB=1b26b3dca87c03702b1dc11207d7b72a 2023-05-10 04:55:20,092 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-10 04:55:20,092 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ba55ae97fe6d26f2fd2a296837570536, daughterA=48042d1ae86f00fdead71e71ad8292a9, daughterB=1b26b3dca87c03702b1dc11207d7b72a at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-10 04:55:20,338 INFO [Listener at localhost/45007] procedure.MasterProcedureTestingUtility(454): Restart 6 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ba55ae97fe6d26f2fd2a296837570536, daughterA=48042d1ae86f00fdead71e71ad8292a9, daughterB=1b26b3dca87c03702b1dc11207d7b72a 2023-05-10 04:55:20,338 INFO [Listener at localhost/45007] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-10 04:55:20,338 INFO [Listener at localhost/45007] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-10 04:55:20,338 INFO [Listener at localhost/45007] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-10 04:55:20,338 ERROR [Listener at localhost/45007] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-10 04:55:20,338 INFO [Listener at localhost/45007] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-10 04:55:20,338 INFO [Listener at localhost/45007] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-10 04:55:20,338 INFO [Listener at localhost/45007] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-10 04:55:20,338 INFO [Listener at localhost/45007] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-10 04:55:20,339 INFO [Listener at localhost/45007] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-10 04:55:20,340 DEBUG [Listener at localhost/45007] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-05-10 04:55:20,340 DEBUG [Listener at localhost/45007] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-05-10 04:55:20,340 DEBUG [Listener at localhost/45007] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-05-10 04:55:20,340 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-10 04:55:20,341 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-10 04:55:20,341 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-10 04:55:20,341 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-10 04:55:20,341 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:55:20,341 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=93c2cc329b1fe1893cfbc28c56960522, daughterA=19040b6538a6bdeed7c833cfb44614b6, daughterB=054f69c06f126aad97b290410d5d6188 2023-05-10 04:55:20,341 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:55:20,341 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:55:20,342 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-10 04:55:20,342 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6c70954228a0bbe97dfdc76861367c66, daughterA=75e2592cb2ac08bd9f617bb9d0a0c896, daughterB=b90fd344aff14f0728a80c3d09539e54 2023-05-10 04:55:20,342 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-05-10 04:55:20,342 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-10 04:55:20,342 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-10 04:55:20,342 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:55:20,342 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=4dcb62249031b560a1bc5d03e43ac855, ASSIGN 2023-05-10 04:55:20,342 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-05-10 04:55:20,343 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-05-10 04:55:20,343 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-10 04:55:20,343 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=a7ae61c05a799883e33cfd9dfad56fa1, daughterA=944df92d059326a46982ff7979f8a940, daughterB=fa1eb74e4c5521990fb9118cc6dfd407 2023-05-10 04:55:20,343 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-10 04:55:20,343 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-10 04:55:20,343 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:55:20,343 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=e0a753dfc7b1edf68920ae1543edc38c, daughterA=08557250442eebc7c86235cdf682b1a9, daughterB=e1dcbf091d2db6dbef8d701eb3d89914 2023-05-10 04:55:20,343 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:55:20,343 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:55:20,344 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-10 04:55:20,344 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-10 04:55:20,344 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,37995,1683694419499, table=testRollbackForSplitTableRegionWithReplica, region=bddd3bfc356670bc9cd9492c47d95a01 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=d2b34babccb97d3c1ac066d9e90ae654, daughterA=0141d5003cd2aac0ddef526f1a98709b, daughterB=b919ef6ce9c770ae26c6c5ee0d2ac459 2023-05-10 04:55:20,344 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-10 04:55:20,344 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-10 04:55:20,345 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-05-10 04:55:20,345 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-05-10 04:55:20,345 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-05-10 04:55:20,345 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-05-10 04:55:20,346 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ba55ae97fe6d26f2fd2a296837570536, daughterA=48042d1ae86f00fdead71e71ad8292a9, daughterB=1b26b3dca87c03702b1dc11207d7b72a 2023-05-10 04:55:20,346 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=ba55ae97fe6d26f2fd2a296837570536, UNASSIGN 2023-05-10 04:55:20,346 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure ba55ae97fe6d26f2fd2a296837570536, server=jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:20,346 DEBUG [Listener at localhost/45007] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ba55ae97fe6d26f2fd2a296837570536, daughterA=48042d1ae86f00fdead71e71ad8292a9, daughterB=1b26b3dca87c03702b1dc11207d7b72a held the lock before restarting, call acquireLock to restore it. 2023-05-10 04:55:20,346 INFO [Listener at localhost/45007] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ba55ae97fe6d26f2fd2a296837570536, daughterA=48042d1ae86f00fdead71e71ad8292a9, daughterB=1b26b3dca87c03702b1dc11207d7b72a 2023-05-10 04:55:20,346 INFO [Listener at localhost/45007] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ba55ae97fe6d26f2fd2a296837570536, daughterA=48042d1ae86f00fdead71e71ad8292a9, daughterB=1b26b3dca87c03702b1dc11207d7b72a 2023-05-10 04:55:20,346 INFO [Listener at localhost/45007] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ba55ae97fe6d26f2fd2a296837570536, daughterA=48042d1ae86f00fdead71e71ad8292a9, daughterB=1b26b3dca87c03702b1dc11207d7b72a 2023-05-10 04:55:20,346 DEBUG [Listener at localhost/45007] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=ba55ae97fe6d26f2fd2a296837570536, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-05-10 04:55:20,346 DEBUG [Listener at localhost/45007] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure ba55ae97fe6d26f2fd2a296837570536, server=jenkins-hbase4.apache.org,36665,1683694419452 didn't hold the lock before restarting, skip acquiring lock. 2023-05-10 04:55:20,346 INFO [Listener at localhost/45007] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-05-10 04:55:20,347 DEBUG [Listener at localhost/45007] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-10 04:55:20,347 DEBUG [Listener at localhost/45007] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,37995,1683694419499, table=hbase:meta, region=1588230740 2023-05-10 04:55:20,348 INFO [Listener at localhost/45007] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683694550348 2023-05-10 04:55:20,349 DEBUG [Listener at localhost/45007] assignment.AssignmentManager(1517): Joining cluster... 2023-05-10 04:55:20,349 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ba55ae97fe6d26f2fd2a296837570536, daughterA=48042d1ae86f00fdead71e71ad8292a9, daughterB=1b26b3dca87c03702b1dc11207d7b72a 2023-05-10 04:55:20,351 DEBUG [Listener at localhost/45007] assignment.RegionStateStore(147): Load hbase:meta entry region=ccfda81dc2f13ffc1429bef67d349028, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36665,1683694419452, regionLocation=jenkins-hbase4.apache.org,36665,1683694419452, openSeqNum=2 2023-05-10 04:55:20,351 DEBUG [Listener at localhost/45007] assignment.RegionStateStore(147): Load hbase:meta entry region=ba55ae97fe6d26f2fd2a296837570536, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,36665,1683694419452, regionLocation=jenkins-hbase4.apache.org,36665,1683694419452, openSeqNum=2 2023-05-10 04:55:20,351 INFO [Listener at localhost/45007] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-10 04:55:20,352 INFO [Listener at localhost/45007] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683694580352 2023-05-10 04:55:20,352 INFO [Listener at localhost/45007] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683694640352 2023-05-10 04:55:20,352 INFO [Listener at localhost/45007] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-05-10 04:55:20,352 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-10 04:55:20,353 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1683694516443.ba55ae97fe6d26f2fd2a296837570536.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1683694520353"},{"qualifier":"splitA","vlen":66,"tag":[],"timestamp":"1683694520353"},{"qualifier":"splitB","vlen":66,"tag":[],"timestamp":"1683694520353"}]},"ts":"1683694520353"} 2023-05-10 04:55:20,354 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1683694518241.48042d1ae86f00fdead71e71ad8292a9.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1683694520353"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683694520353"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683694520353"}]},"ts":"1683694520353"} 2023-05-10 04:55:20,354 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1683694518241.1b26b3dca87c03702b1dc11207d7b72a.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1683694520353"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683694520353"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683694520353"}]},"ts":"1683694520353"} 2023-05-10 04:55:20,355 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-10 04:55:20,355 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ba55ae97fe6d26f2fd2a296837570536, daughterA=48042d1ae86f00fdead71e71ad8292a9, daughterB=1b26b3dca87c03702b1dc11207d7b72a 2023-05-10 04:55:20,355 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-10 04:55:20,355 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ba55ae97fe6d26f2fd2a296837570536, daughterA=48042d1ae86f00fdead71e71ad8292a9, daughterB=1b26b3dca87c03702b1dc11207d7b72a at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-10 04:55:20,602 INFO [Listener at localhost/45007] procedure.MasterProcedureTestingUtility(454): Restart 7 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ba55ae97fe6d26f2fd2a296837570536, daughterA=48042d1ae86f00fdead71e71ad8292a9, daughterB=1b26b3dca87c03702b1dc11207d7b72a 2023-05-10 04:55:20,602 INFO [Listener at localhost/45007] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-10 04:55:20,602 INFO [Listener at localhost/45007] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-10 04:55:20,602 INFO [Listener at localhost/45007] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-10 04:55:20,602 ERROR [Listener at localhost/45007] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-10 04:55:20,602 INFO [Listener at localhost/45007] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-10 04:55:20,602 INFO [Listener at localhost/45007] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-10 04:55:20,602 INFO [Listener at localhost/45007] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-10 04:55:20,603 INFO [Listener at localhost/45007] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-10 04:55:20,603 INFO [Listener at localhost/45007] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-10 04:55:20,604 DEBUG [Listener at localhost/45007] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10] 2023-05-10 04:55:20,604 DEBUG [Listener at localhost/45007] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-05-10 04:55:20,604 DEBUG [Listener at localhost/45007] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-05-10 04:55:20,605 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-10 04:55:20,605 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-10 04:55:20,605 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-10 04:55:20,605 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-10 04:55:20,605 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:55:20,606 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=93c2cc329b1fe1893cfbc28c56960522, daughterA=19040b6538a6bdeed7c833cfb44614b6, daughterB=054f69c06f126aad97b290410d5d6188 2023-05-10 04:55:20,606 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:55:20,606 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:55:20,606 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-10 04:55:20,606 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6c70954228a0bbe97dfdc76861367c66, daughterA=75e2592cb2ac08bd9f617bb9d0a0c896, daughterB=b90fd344aff14f0728a80c3d09539e54 2023-05-10 04:55:20,606 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-05-10 04:55:20,606 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-10 04:55:20,606 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-10 04:55:20,607 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:55:20,607 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=4dcb62249031b560a1bc5d03e43ac855, ASSIGN 2023-05-10 04:55:20,607 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-05-10 04:55:20,607 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-05-10 04:55:20,607 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-10 04:55:20,608 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=a7ae61c05a799883e33cfd9dfad56fa1, daughterA=944df92d059326a46982ff7979f8a940, daughterB=fa1eb74e4c5521990fb9118cc6dfd407 2023-05-10 04:55:20,608 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-10 04:55:20,608 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-10 04:55:20,608 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:55:20,608 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=e0a753dfc7b1edf68920ae1543edc38c, daughterA=08557250442eebc7c86235cdf682b1a9, daughterB=e1dcbf091d2db6dbef8d701eb3d89914 2023-05-10 04:55:20,608 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:55:20,608 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:55:20,608 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-10 04:55:20,609 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-10 04:55:20,609 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,37995,1683694419499, table=testRollbackForSplitTableRegionWithReplica, region=bddd3bfc356670bc9cd9492c47d95a01 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=d2b34babccb97d3c1ac066d9e90ae654, daughterA=0141d5003cd2aac0ddef526f1a98709b, daughterB=b919ef6ce9c770ae26c6c5ee0d2ac459 2023-05-10 04:55:20,609 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-10 04:55:20,609 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-10 04:55:20,609 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-05-10 04:55:20,609 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-05-10 04:55:20,610 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-05-10 04:55:20,610 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-05-10 04:55:20,610 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ba55ae97fe6d26f2fd2a296837570536, daughterA=48042d1ae86f00fdead71e71ad8292a9, daughterB=1b26b3dca87c03702b1dc11207d7b72a 2023-05-10 04:55:20,610 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=ba55ae97fe6d26f2fd2a296837570536, UNASSIGN 2023-05-10 04:55:20,610 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure ba55ae97fe6d26f2fd2a296837570536, server=jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:20,610 DEBUG [Listener at localhost/45007] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ba55ae97fe6d26f2fd2a296837570536, daughterA=48042d1ae86f00fdead71e71ad8292a9, daughterB=1b26b3dca87c03702b1dc11207d7b72a held the lock before restarting, call acquireLock to restore it. 2023-05-10 04:55:20,610 INFO [Listener at localhost/45007] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ba55ae97fe6d26f2fd2a296837570536, daughterA=48042d1ae86f00fdead71e71ad8292a9, daughterB=1b26b3dca87c03702b1dc11207d7b72a 2023-05-10 04:55:20,610 INFO [Listener at localhost/45007] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ba55ae97fe6d26f2fd2a296837570536, daughterA=48042d1ae86f00fdead71e71ad8292a9, daughterB=1b26b3dca87c03702b1dc11207d7b72a 2023-05-10 04:55:20,610 INFO [Listener at localhost/45007] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ba55ae97fe6d26f2fd2a296837570536, daughterA=48042d1ae86f00fdead71e71ad8292a9, daughterB=1b26b3dca87c03702b1dc11207d7b72a 2023-05-10 04:55:20,610 DEBUG [Listener at localhost/45007] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=ba55ae97fe6d26f2fd2a296837570536, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-05-10 04:55:20,610 DEBUG [Listener at localhost/45007] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure ba55ae97fe6d26f2fd2a296837570536, server=jenkins-hbase4.apache.org,36665,1683694419452 didn't hold the lock before restarting, skip acquiring lock. 2023-05-10 04:55:20,611 INFO [Listener at localhost/45007] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-05-10 04:55:20,611 DEBUG [Listener at localhost/45007] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-10 04:55:20,612 DEBUG [Listener at localhost/45007] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,37995,1683694419499, table=hbase:meta, region=1588230740 2023-05-10 04:55:20,612 INFO [Listener at localhost/45007] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683694550612 2023-05-10 04:55:20,614 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ba55ae97fe6d26f2fd2a296837570536, daughterA=48042d1ae86f00fdead71e71ad8292a9, daughterB=1b26b3dca87c03702b1dc11207d7b72a 2023-05-10 04:55:20,614 DEBUG [Listener at localhost/45007] assignment.AssignmentManager(1517): Joining cluster... 2023-05-10 04:55:20,617 DEBUG [Listener at localhost/45007] assignment.RegionStateStore(147): Load hbase:meta entry region=ccfda81dc2f13ffc1429bef67d349028, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36665,1683694419452, regionLocation=jenkins-hbase4.apache.org,36665,1683694419452, openSeqNum=2 2023-05-10 04:55:20,617 DEBUG [Listener at localhost/45007] assignment.RegionStateStore(147): Load hbase:meta entry region=ba55ae97fe6d26f2fd2a296837570536, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,36665,1683694419452, regionLocation=jenkins-hbase4.apache.org,36665,1683694419452, openSeqNum=2 2023-05-10 04:55:20,618 INFO [Listener at localhost/45007] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase4.apache.org,36665,1683694419452, table=testRecoveryAndDoubleExecution, region=ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:20,618 DEBUG [Listener at localhost/45007] assignment.RegionStateStore(147): Load hbase:meta entry region=48042d1ae86f00fdead71e71ad8292a9, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-05-10 04:55:20,618 DEBUG [Listener at localhost/45007] assignment.RegionStateStore(147): Load hbase:meta entry region=1b26b3dca87c03702b1dc11207d7b72a, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-05-10 04:55:20,618 INFO [Listener at localhost/45007] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-10 04:55:20,618 INFO [Listener at localhost/45007] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683694580618 2023-05-10 04:55:20,618 INFO [Listener at localhost/45007] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683694640618 2023-05-10 04:55:20,618 INFO [Listener at localhost/45007] assignment.AssignmentManager(1536): Joined the cluster in 4 msec 2023-05-10 04:55:20,619 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1683694516443.ba55ae97fe6d26f2fd2a296837570536.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1683694520619"},{"qualifier":"splitA","vlen":66,"tag":[],"timestamp":"1683694520619"},{"qualifier":"splitB","vlen":66,"tag":[],"timestamp":"1683694520619"}]},"ts":"1683694520619"} 2023-05-10 04:55:20,619 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1683694518241.48042d1ae86f00fdead71e71ad8292a9.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1683694520619"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683694520619"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683694520619"}]},"ts":"1683694520619"} 2023-05-10 04:55:20,619 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1683694518241.1b26b3dca87c03702b1dc11207d7b72a.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1683694520619"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683694520619"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683694520619"}]},"ts":"1683694520619"} 2023-05-10 04:55:20,621 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-10 04:55:20,623 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-10 04:55:20,623 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ba55ae97fe6d26f2fd2a296837570536, daughterA=48042d1ae86f00fdead71e71ad8292a9, daughterB=1b26b3dca87c03702b1dc11207d7b72a 2023-05-10 04:55:20,623 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-10 04:55:20,623 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ba55ae97fe6d26f2fd2a296837570536, daughterA=48042d1ae86f00fdead71e71ad8292a9, daughterB=1b26b3dca87c03702b1dc11207d7b72a at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-10 04:55:20,869 INFO [Listener at localhost/45007] procedure.MasterProcedureTestingUtility(454): Restart 8 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ba55ae97fe6d26f2fd2a296837570536, daughterA=48042d1ae86f00fdead71e71ad8292a9, daughterB=1b26b3dca87c03702b1dc11207d7b72a 2023-05-10 04:55:20,869 INFO [Listener at localhost/45007] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-10 04:55:20,869 INFO [Listener at localhost/45007] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-10 04:55:20,869 INFO [Listener at localhost/45007] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-10 04:55:20,869 ERROR [Listener at localhost/45007] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-10 04:55:20,870 INFO [Listener at localhost/45007] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-10 04:55:20,870 INFO [Listener at localhost/45007] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-10 04:55:20,870 INFO [Listener at localhost/45007] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-10 04:55:20,870 INFO [Listener at localhost/45007] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-10 04:55:20,871 INFO [Listener at localhost/45007] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-10 04:55:20,872 DEBUG [Listener at localhost/45007] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10, 11] 2023-05-10 04:55:20,872 DEBUG [Listener at localhost/45007] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-05-10 04:55:20,873 DEBUG [Listener at localhost/45007] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-05-10 04:55:20,873 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-10 04:55:20,873 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-10 04:55:20,874 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-10 04:55:20,874 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-10 04:55:20,874 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:55:20,874 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=93c2cc329b1fe1893cfbc28c56960522, daughterA=19040b6538a6bdeed7c833cfb44614b6, daughterB=054f69c06f126aad97b290410d5d6188 2023-05-10 04:55:20,875 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:55:20,875 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:55:20,875 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-10 04:55:20,875 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6c70954228a0bbe97dfdc76861367c66, daughterA=75e2592cb2ac08bd9f617bb9d0a0c896, daughterB=b90fd344aff14f0728a80c3d09539e54 2023-05-10 04:55:20,875 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-05-10 04:55:20,876 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-10 04:55:20,876 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-10 04:55:20,876 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:55:20,876 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=4dcb62249031b560a1bc5d03e43ac855, ASSIGN 2023-05-10 04:55:20,876 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-05-10 04:55:20,877 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-05-10 04:55:20,877 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-10 04:55:20,877 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=a7ae61c05a799883e33cfd9dfad56fa1, daughterA=944df92d059326a46982ff7979f8a940, daughterB=fa1eb74e4c5521990fb9118cc6dfd407 2023-05-10 04:55:20,877 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-10 04:55:20,877 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-10 04:55:20,878 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:55:20,878 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=e0a753dfc7b1edf68920ae1543edc38c, daughterA=08557250442eebc7c86235cdf682b1a9, daughterB=e1dcbf091d2db6dbef8d701eb3d89914 2023-05-10 04:55:20,878 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:55:20,878 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:55:20,878 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-10 04:55:20,879 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-10 04:55:20,879 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,37995,1683694419499, table=testRollbackForSplitTableRegionWithReplica, region=bddd3bfc356670bc9cd9492c47d95a01 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=d2b34babccb97d3c1ac066d9e90ae654, daughterA=0141d5003cd2aac0ddef526f1a98709b, daughterB=b919ef6ce9c770ae26c6c5ee0d2ac459 2023-05-10 04:55:20,879 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-10 04:55:20,879 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-10 04:55:20,879 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-05-10 04:55:20,880 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-05-10 04:55:20,880 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-05-10 04:55:20,880 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-05-10 04:55:20,880 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ba55ae97fe6d26f2fd2a296837570536, daughterA=48042d1ae86f00fdead71e71ad8292a9, daughterB=1b26b3dca87c03702b1dc11207d7b72a 2023-05-10 04:55:20,881 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=ba55ae97fe6d26f2fd2a296837570536, UNASSIGN 2023-05-10 04:55:20,881 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure ba55ae97fe6d26f2fd2a296837570536, server=jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:20,881 DEBUG [Listener at localhost/45007] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ba55ae97fe6d26f2fd2a296837570536, daughterA=48042d1ae86f00fdead71e71ad8292a9, daughterB=1b26b3dca87c03702b1dc11207d7b72a held the lock before restarting, call acquireLock to restore it. 2023-05-10 04:55:20,881 INFO [Listener at localhost/45007] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ba55ae97fe6d26f2fd2a296837570536, daughterA=48042d1ae86f00fdead71e71ad8292a9, daughterB=1b26b3dca87c03702b1dc11207d7b72a 2023-05-10 04:55:20,881 INFO [Listener at localhost/45007] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ba55ae97fe6d26f2fd2a296837570536, daughterA=48042d1ae86f00fdead71e71ad8292a9, daughterB=1b26b3dca87c03702b1dc11207d7b72a 2023-05-10 04:55:20,881 INFO [Listener at localhost/45007] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ba55ae97fe6d26f2fd2a296837570536, daughterA=48042d1ae86f00fdead71e71ad8292a9, daughterB=1b26b3dca87c03702b1dc11207d7b72a 2023-05-10 04:55:20,881 DEBUG [Listener at localhost/45007] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=ba55ae97fe6d26f2fd2a296837570536, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-05-10 04:55:20,881 DEBUG [Listener at localhost/45007] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure ba55ae97fe6d26f2fd2a296837570536, server=jenkins-hbase4.apache.org,36665,1683694419452 didn't hold the lock before restarting, skip acquiring lock. 2023-05-10 04:55:20,882 INFO [Listener at localhost/45007] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 10 msec 2023-05-10 04:55:20,882 DEBUG [Listener at localhost/45007] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-10 04:55:20,883 DEBUG [Listener at localhost/45007] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,37995,1683694419499, table=hbase:meta, region=1588230740 2023-05-10 04:55:20,886 INFO [Listener at localhost/45007] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683694550886 2023-05-10 04:55:20,888 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ba55ae97fe6d26f2fd2a296837570536, daughterA=48042d1ae86f00fdead71e71ad8292a9, daughterB=1b26b3dca87c03702b1dc11207d7b72a 2023-05-10 04:55:20,888 DEBUG [Listener at localhost/45007] assignment.AssignmentManager(1517): Joining cluster... 2023-05-10 04:55:20,891 DEBUG [Listener at localhost/45007] assignment.RegionStateStore(147): Load hbase:meta entry region=ccfda81dc2f13ffc1429bef67d349028, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36665,1683694419452, regionLocation=jenkins-hbase4.apache.org,36665,1683694419452, openSeqNum=2 2023-05-10 04:55:20,891 DEBUG [Listener at localhost/45007] assignment.RegionStateStore(147): Load hbase:meta entry region=ba55ae97fe6d26f2fd2a296837570536, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,36665,1683694419452, regionLocation=jenkins-hbase4.apache.org,36665,1683694419452, openSeqNum=2 2023-05-10 04:55:20,891 INFO [Listener at localhost/45007] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase4.apache.org,36665,1683694419452, table=testRecoveryAndDoubleExecution, region=ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:20,891 DEBUG [Listener at localhost/45007] assignment.RegionStateStore(147): Load hbase:meta entry region=48042d1ae86f00fdead71e71ad8292a9, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-05-10 04:55:20,892 DEBUG [Listener at localhost/45007] assignment.RegionStateStore(147): Load hbase:meta entry region=1b26b3dca87c03702b1dc11207d7b72a, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-05-10 04:55:20,892 INFO [Listener at localhost/45007] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-10 04:55:20,892 INFO [Listener at localhost/45007] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683694580892 2023-05-10 04:55:20,892 INFO [Listener at localhost/45007] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683694640892 2023-05-10 04:55:20,892 INFO [Listener at localhost/45007] assignment.AssignmentManager(1536): Joined the cluster in 4 msec 2023-05-10 04:55:20,892 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-10 04:55:20,894 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=48042d1ae86f00fdead71e71ad8292a9, ASSIGN}, {pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=1b26b3dca87c03702b1dc11207d7b72a, ASSIGN}] 2023-05-10 04:55:20,894 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-10 04:55:20,894 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGION_POST_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ba55ae97fe6d26f2fd2a296837570536, daughterA=48042d1ae86f00fdead71e71ad8292a9, daughterB=1b26b3dca87c03702b1dc11207d7b72a 2023-05-10 04:55:20,894 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-10 04:55:20,894 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGION_POST_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ba55ae97fe6d26f2fd2a296837570536, daughterA=48042d1ae86f00fdead71e71ad8292a9, daughterB=1b26b3dca87c03702b1dc11207d7b72a at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-10 04:55:21,142 INFO [Listener at localhost/45007] procedure.MasterProcedureTestingUtility(454): Restart 9 exec state=pid=112, state=WAITING:SPLIT_TABLE_REGION_POST_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ba55ae97fe6d26f2fd2a296837570536, daughterA=48042d1ae86f00fdead71e71ad8292a9, daughterB=1b26b3dca87c03702b1dc11207d7b72a 2023-05-10 04:55:21,142 INFO [Listener at localhost/45007] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-10 04:55:21,142 INFO [Listener at localhost/45007] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-10 04:55:21,143 INFO [Listener at localhost/45007] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-10 04:55:21,143 ERROR [Listener at localhost/45007] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-10 04:55:21,143 INFO [Listener at localhost/45007] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-10 04:55:21,143 INFO [Listener at localhost/45007] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-10 04:55:21,143 INFO [Listener at localhost/45007] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-10 04:55:21,143 INFO [Listener at localhost/45007] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-10 04:55:21,144 INFO [Listener at localhost/45007] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-10 04:55:21,145 DEBUG [Listener at localhost/45007] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10, 11, 12] 2023-05-10 04:55:21,145 DEBUG [Listener at localhost/45007] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-05-10 04:55:21,145 DEBUG [Listener at localhost/45007] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-05-10 04:55:21,145 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-10 04:55:21,146 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-10 04:55:21,146 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-10 04:55:21,146 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-10 04:55:21,146 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:55:21,146 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=93c2cc329b1fe1893cfbc28c56960522, daughterA=19040b6538a6bdeed7c833cfb44614b6, daughterB=054f69c06f126aad97b290410d5d6188 2023-05-10 04:55:21,146 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:55:21,147 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:55:21,147 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-10 04:55:21,147 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6c70954228a0bbe97dfdc76861367c66, daughterA=75e2592cb2ac08bd9f617bb9d0a0c896, daughterB=b90fd344aff14f0728a80c3d09539e54 2023-05-10 04:55:21,147 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-05-10 04:55:21,147 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-10 04:55:21,147 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-10 04:55:21,147 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:55:21,147 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=4dcb62249031b560a1bc5d03e43ac855, ASSIGN 2023-05-10 04:55:21,148 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-05-10 04:55:21,148 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-05-10 04:55:21,148 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-10 04:55:21,148 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=a7ae61c05a799883e33cfd9dfad56fa1, daughterA=944df92d059326a46982ff7979f8a940, daughterB=fa1eb74e4c5521990fb9118cc6dfd407 2023-05-10 04:55:21,148 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-10 04:55:21,148 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-10 04:55:21,148 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:55:21,148 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=e0a753dfc7b1edf68920ae1543edc38c, daughterA=08557250442eebc7c86235cdf682b1a9, daughterB=e1dcbf091d2db6dbef8d701eb3d89914 2023-05-10 04:55:21,148 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:55:21,148 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:55:21,149 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-10 04:55:21,149 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-10 04:55:21,149 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,37995,1683694419499, table=testRollbackForSplitTableRegionWithReplica, region=bddd3bfc356670bc9cd9492c47d95a01 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=d2b34babccb97d3c1ac066d9e90ae654, daughterA=0141d5003cd2aac0ddef526f1a98709b, daughterB=b919ef6ce9c770ae26c6c5ee0d2ac459 2023-05-10 04:55:21,149 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-10 04:55:21,149 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-10 04:55:21,150 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-05-10 04:55:21,150 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-05-10 04:55:21,150 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-05-10 04:55:21,150 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-05-10 04:55:21,150 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ba55ae97fe6d26f2fd2a296837570536, daughterA=48042d1ae86f00fdead71e71ad8292a9, daughterB=1b26b3dca87c03702b1dc11207d7b72a 2023-05-10 04:55:21,150 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=ba55ae97fe6d26f2fd2a296837570536, UNASSIGN 2023-05-10 04:55:21,150 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure ba55ae97fe6d26f2fd2a296837570536, server=jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:21,151 DEBUG [Listener at localhost/45007] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ba55ae97fe6d26f2fd2a296837570536, daughterA=48042d1ae86f00fdead71e71ad8292a9, daughterB=1b26b3dca87c03702b1dc11207d7b72a held the lock before restarting, call acquireLock to restore it. 2023-05-10 04:55:21,151 INFO [Listener at localhost/45007] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ba55ae97fe6d26f2fd2a296837570536, daughterA=48042d1ae86f00fdead71e71ad8292a9, daughterB=1b26b3dca87c03702b1dc11207d7b72a 2023-05-10 04:55:21,151 INFO [Listener at localhost/45007] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ba55ae97fe6d26f2fd2a296837570536, daughterA=48042d1ae86f00fdead71e71ad8292a9, daughterB=1b26b3dca87c03702b1dc11207d7b72a 2023-05-10 04:55:21,151 INFO [Listener at localhost/45007] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ba55ae97fe6d26f2fd2a296837570536, daughterA=48042d1ae86f00fdead71e71ad8292a9, daughterB=1b26b3dca87c03702b1dc11207d7b72a 2023-05-10 04:55:21,151 DEBUG [Listener at localhost/45007] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=ba55ae97fe6d26f2fd2a296837570536, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-05-10 04:55:21,151 DEBUG [Listener at localhost/45007] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure ba55ae97fe6d26f2fd2a296837570536, server=jenkins-hbase4.apache.org,36665,1683694419452 didn't hold the lock before restarting, skip acquiring lock. 2023-05-10 04:55:21,151 INFO [Listener at localhost/45007] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-05-10 04:55:21,152 DEBUG [Listener at localhost/45007] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-10 04:55:21,152 DEBUG [Listener at localhost/45007] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,37995,1683694419499, table=hbase:meta, region=1588230740 2023-05-10 04:55:21,152 INFO [Listener at localhost/45007] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683694551152 2023-05-10 04:55:21,154 DEBUG [Listener at localhost/45007] assignment.AssignmentManager(1517): Joining cluster... 2023-05-10 04:55:21,154 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ba55ae97fe6d26f2fd2a296837570536, daughterA=48042d1ae86f00fdead71e71ad8292a9, daughterB=1b26b3dca87c03702b1dc11207d7b72a 2023-05-10 04:55:21,157 DEBUG [Listener at localhost/45007] assignment.RegionStateStore(147): Load hbase:meta entry region=ccfda81dc2f13ffc1429bef67d349028, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36665,1683694419452, regionLocation=jenkins-hbase4.apache.org,36665,1683694419452, openSeqNum=2 2023-05-10 04:55:21,157 DEBUG [Listener at localhost/45007] assignment.RegionStateStore(147): Load hbase:meta entry region=ba55ae97fe6d26f2fd2a296837570536, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,36665,1683694419452, regionLocation=jenkins-hbase4.apache.org,36665,1683694419452, openSeqNum=2 2023-05-10 04:55:21,157 INFO [Listener at localhost/45007] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase4.apache.org,36665,1683694419452, table=testRecoveryAndDoubleExecution, region=ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:21,157 DEBUG [Listener at localhost/45007] assignment.RegionStateStore(147): Load hbase:meta entry region=48042d1ae86f00fdead71e71ad8292a9, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-05-10 04:55:21,157 DEBUG [Listener at localhost/45007] assignment.RegionStateStore(147): Load hbase:meta entry region=1b26b3dca87c03702b1dc11207d7b72a, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-05-10 04:55:21,158 INFO [Listener at localhost/45007] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-10 04:55:21,158 INFO [Listener at localhost/45007] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683694581158 2023-05-10 04:55:21,158 INFO [Listener at localhost/45007] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683694641158 2023-05-10 04:55:21,158 INFO [Listener at localhost/45007] assignment.AssignmentManager(1536): Joined the cluster in 4 msec 2023-05-10 04:55:21,158 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=48042d1ae86f00fdead71e71ad8292a9, ASSIGN}, {pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=1b26b3dca87c03702b1dc11207d7b72a, ASSIGN}] 2023-05-10 04:55:21,158 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-10 04:55:21,160 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=1b26b3dca87c03702b1dc11207d7b72a, ASSIGN 2023-05-10 04:55:21,161 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=1b26b3dca87c03702b1dc11207d7b72a, ASSIGN; state=CLOSED, location=jenkins-hbase4.apache.org,36665,1683694419452; forceNewPlan=false, retain=false 2023-05-10 04:55:21,161 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=48042d1ae86f00fdead71e71ad8292a9, ASSIGN 2023-05-10 04:55:21,162 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=48042d1ae86f00fdead71e71ad8292a9, ASSIGN; state=CLOSED, location=jenkins-hbase4.apache.org,36665,1683694419452; forceNewPlan=false, retain=false 2023-05-10 04:55:21,311 INFO [jenkins-hbase4:37903] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-05-10 04:55:21,313 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=115 updating hbase:meta row=48042d1ae86f00fdead71e71ad8292a9, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:21,313 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1683694518241.48042d1ae86f00fdead71e71ad8292a9.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1683694521313"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683694521313"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683694521313"}]},"ts":"1683694521313"} 2023-05-10 04:55:21,314 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=117, ppid=115, state=RUNNABLE; OpenRegionProcedure 48042d1ae86f00fdead71e71ad8292a9, server=jenkins-hbase4.apache.org,36665,1683694419452}] 2023-05-10 04:55:21,316 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=116 updating hbase:meta row=1b26b3dca87c03702b1dc11207d7b72a, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:21,317 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1683694518241.1b26b3dca87c03702b1dc11207d7b72a.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1683694521316"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683694521316"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683694521316"}]},"ts":"1683694521316"} 2023-05-10 04:55:21,318 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=118, ppid=116, state=RUNNABLE; OpenRegionProcedure 1b26b3dca87c03702b1dc11207d7b72a, server=jenkins-hbase4.apache.org,36665,1683694419452}] 2023-05-10 04:55:21,468 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testRecoveryAndDoubleExecution,,1683694518241.48042d1ae86f00fdead71e71ad8292a9. 2023-05-10 04:55:21,469 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 48042d1ae86f00fdead71e71ad8292a9, NAME => 'testRecoveryAndDoubleExecution,,1683694518241.48042d1ae86f00fdead71e71ad8292a9.', STARTKEY => '', ENDKEY => '41'} 2023-05-10 04:55:21,469 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-05-10 04:55:21,469 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRecoveryAndDoubleExecution 48042d1ae86f00fdead71e71ad8292a9 2023-05-10 04:55:21,469 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testRecoveryAndDoubleExecution,,1683694518241.48042d1ae86f00fdead71e71ad8292a9.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-10 04:55:21,469 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 48042d1ae86f00fdead71e71ad8292a9 2023-05-10 04:55:21,469 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 48042d1ae86f00fdead71e71ad8292a9 2023-05-10 04:55:21,471 INFO [StoreOpener-48042d1ae86f00fdead71e71ad8292a9-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 48042d1ae86f00fdead71e71ad8292a9 2023-05-10 04:55:21,471 DEBUG [StoreOpener-48042d1ae86f00fdead71e71ad8292a9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/cf1 2023-05-10 04:55:21,471 DEBUG [StoreOpener-48042d1ae86f00fdead71e71ad8292a9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/cf1 2023-05-10 04:55:21,472 INFO [StoreOpener-48042d1ae86f00fdead71e71ad8292a9-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 48042d1ae86f00fdead71e71ad8292a9 columnFamilyName cf1 2023-05-10 04:55:21,483 DEBUG [StoreOpener-48042d1ae86f00fdead71e71ad8292a9-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/cf1/17ee8f61ba9d4f678887f8a52b767dbb.ba55ae97fe6d26f2fd2a296837570536->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf1/17ee8f61ba9d4f678887f8a52b767dbb-bottom 2023-05-10 04:55:21,486 DEBUG [StoreOpener-48042d1ae86f00fdead71e71ad8292a9-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/cf1/3488050453f04078bc1365497a2e3f88.ba55ae97fe6d26f2fd2a296837570536->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf1/3488050453f04078bc1365497a2e3f88-bottom 2023-05-10 04:55:21,490 DEBUG [StoreOpener-48042d1ae86f00fdead71e71ad8292a9-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/cf1/4e150d9797e44e02a343bd11ff7b3dc3.ba55ae97fe6d26f2fd2a296837570536->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf1/4e150d9797e44e02a343bd11ff7b3dc3-bottom 2023-05-10 04:55:21,493 DEBUG [StoreOpener-48042d1ae86f00fdead71e71ad8292a9-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/cf1/aa810537cf12426fa1bcdd14225b68ee.ba55ae97fe6d26f2fd2a296837570536->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf1/aa810537cf12426fa1bcdd14225b68ee-bottom 2023-05-10 04:55:21,497 DEBUG [StoreOpener-48042d1ae86f00fdead71e71ad8292a9-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/cf1/b3e182af00c84ddb81eb7331183b802e.ba55ae97fe6d26f2fd2a296837570536->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf1/b3e182af00c84ddb81eb7331183b802e-bottom 2023-05-10 04:55:21,500 DEBUG [StoreOpener-48042d1ae86f00fdead71e71ad8292a9-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/cf1/bd5a17faa7564deb857bb9ec28719ee6.ba55ae97fe6d26f2fd2a296837570536->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf1/bd5a17faa7564deb857bb9ec28719ee6-bottom 2023-05-10 04:55:21,503 DEBUG [StoreOpener-48042d1ae86f00fdead71e71ad8292a9-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/cf1/c2d599f271fb497b8b272a3bfd391f6e.ba55ae97fe6d26f2fd2a296837570536->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf1/c2d599f271fb497b8b272a3bfd391f6e-bottom 2023-05-10 04:55:21,503 INFO [StoreOpener-48042d1ae86f00fdead71e71ad8292a9-1] regionserver.HStore(310): Store=48042d1ae86f00fdead71e71ad8292a9/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-10 04:55:21,503 INFO [StoreOpener-48042d1ae86f00fdead71e71ad8292a9-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 48042d1ae86f00fdead71e71ad8292a9 2023-05-10 04:55:21,504 DEBUG [StoreOpener-48042d1ae86f00fdead71e71ad8292a9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/cf2 2023-05-10 04:55:21,504 DEBUG [StoreOpener-48042d1ae86f00fdead71e71ad8292a9-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/cf2 2023-05-10 04:55:21,504 INFO [StoreOpener-48042d1ae86f00fdead71e71ad8292a9-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 48042d1ae86f00fdead71e71ad8292a9 columnFamilyName cf2 2023-05-10 04:55:21,516 DEBUG [StoreOpener-48042d1ae86f00fdead71e71ad8292a9-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/cf2/14e73a6c8cd44ed2ab40f783dad882ca.ba55ae97fe6d26f2fd2a296837570536->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf2/14e73a6c8cd44ed2ab40f783dad882ca-bottom 2023-05-10 04:55:21,519 DEBUG [StoreOpener-48042d1ae86f00fdead71e71ad8292a9-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/cf2/1747520752eb459e951b7491aa7c62a3.ba55ae97fe6d26f2fd2a296837570536->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf2/1747520752eb459e951b7491aa7c62a3-bottom 2023-05-10 04:55:21,523 DEBUG [StoreOpener-48042d1ae86f00fdead71e71ad8292a9-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/cf2/34f3b8d7d8774320929c1c1bbcc1f132.ba55ae97fe6d26f2fd2a296837570536->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf2/34f3b8d7d8774320929c1c1bbcc1f132-bottom 2023-05-10 04:55:21,526 DEBUG [StoreOpener-48042d1ae86f00fdead71e71ad8292a9-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/cf2/c332f2f6f5a44943a5e8c97725d45512.ba55ae97fe6d26f2fd2a296837570536->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf2/c332f2f6f5a44943a5e8c97725d45512-bottom 2023-05-10 04:55:21,530 DEBUG [StoreOpener-48042d1ae86f00fdead71e71ad8292a9-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/cf2/c4912c4098564070a195e3fd918ecec2.ba55ae97fe6d26f2fd2a296837570536->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf2/c4912c4098564070a195e3fd918ecec2-bottom 2023-05-10 04:55:21,533 DEBUG [StoreOpener-48042d1ae86f00fdead71e71ad8292a9-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/cf2/c845912c71644277a0aed468d0621d59.ba55ae97fe6d26f2fd2a296837570536->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf2/c845912c71644277a0aed468d0621d59-bottom 2023-05-10 04:55:21,537 DEBUG [StoreOpener-48042d1ae86f00fdead71e71ad8292a9-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/cf2/eb76ae13368e4a23855b5680af181f32.ba55ae97fe6d26f2fd2a296837570536->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf2/eb76ae13368e4a23855b5680af181f32-bottom 2023-05-10 04:55:21,537 INFO [StoreOpener-48042d1ae86f00fdead71e71ad8292a9-1] regionserver.HStore(310): Store=48042d1ae86f00fdead71e71ad8292a9/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-10 04:55:21,538 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9 2023-05-10 04:55:21,538 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9 2023-05-10 04:55:21,540 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRecoveryAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-05-10 04:55:21,541 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 48042d1ae86f00fdead71e71ad8292a9 2023-05-10 04:55:21,542 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 48042d1ae86f00fdead71e71ad8292a9; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62150404, jitterRate=-0.07388681173324585}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-05-10 04:55:21,542 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 48042d1ae86f00fdead71e71ad8292a9: 2023-05-10 04:55:21,542 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2334): Post open deploy tasks for testRecoveryAndDoubleExecution,,1683694518241.48042d1ae86f00fdead71e71ad8292a9., pid=117, masterSystemTime=1683694521466 2023-05-10 04:55:21,543 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-10 04:55:21,543 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-05-10 04:55:21,544 DEBUG [RS:0;jenkins-hbase4:36665-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-05-10 04:55:21,544 DEBUG [RS:0;jenkins-hbase4:36665-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-05-10 04:55:21,544 INFO [RS:0;jenkins-hbase4:36665-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,,1683694518241.48042d1ae86f00fdead71e71ad8292a9. 2023-05-10 04:55:21,544 DEBUG [RS:0;jenkins-hbase4:36665-shortCompactions-0] regionserver.HStore(1912): 48042d1ae86f00fdead71e71ad8292a9/cf1 is initiating minor compaction (all files) 2023-05-10 04:55:21,544 INFO [RS:0;jenkins-hbase4:36665-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,,1683694518241.48042d1ae86f00fdead71e71ad8292a9. 2023-05-10 04:55:21,544 DEBUG [RS:0;jenkins-hbase4:36665-longCompactions-0] regionserver.HStore(1912): 48042d1ae86f00fdead71e71ad8292a9/cf2 is initiating minor compaction (all files) 2023-05-10 04:55:21,544 INFO [RS:0;jenkins-hbase4:36665-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 48042d1ae86f00fdead71e71ad8292a9/cf1 in testRecoveryAndDoubleExecution,,1683694518241.48042d1ae86f00fdead71e71ad8292a9. 2023-05-10 04:55:21,544 INFO [RS:0;jenkins-hbase4:36665-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 48042d1ae86f00fdead71e71ad8292a9/cf2 in testRecoveryAndDoubleExecution,,1683694518241.48042d1ae86f00fdead71e71ad8292a9. 2023-05-10 04:55:21,544 INFO [RS:0;jenkins-hbase4:36665-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/cf1/17ee8f61ba9d4f678887f8a52b767dbb.ba55ae97fe6d26f2fd2a296837570536->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf1/17ee8f61ba9d4f678887f8a52b767dbb-bottom, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/cf1/3488050453f04078bc1365497a2e3f88.ba55ae97fe6d26f2fd2a296837570536->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf1/3488050453f04078bc1365497a2e3f88-bottom, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/cf1/c2d599f271fb497b8b272a3bfd391f6e.ba55ae97fe6d26f2fd2a296837570536->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf1/c2d599f271fb497b8b272a3bfd391f6e-bottom, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/cf1/bd5a17faa7564deb857bb9ec28719ee6.ba55ae97fe6d26f2fd2a296837570536->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf1/bd5a17faa7564deb857bb9ec28719ee6-bottom, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/cf1/b3e182af00c84ddb81eb7331183b802e.ba55ae97fe6d26f2fd2a296837570536->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf1/b3e182af00c84ddb81eb7331183b802e-bottom, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/cf1/aa810537cf12426fa1bcdd14225b68ee.ba55ae97fe6d26f2fd2a296837570536->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf1/aa810537cf12426fa1bcdd14225b68ee-bottom, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/cf1/4e150d9797e44e02a343bd11ff7b3dc3.ba55ae97fe6d26f2fd2a296837570536->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf1/4e150d9797e44e02a343bd11ff7b3dc3-bottom] into tmpdir=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/.tmp, totalSize=34.9 K 2023-05-10 04:55:21,544 INFO [RS:0;jenkins-hbase4:36665-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/cf2/eb76ae13368e4a23855b5680af181f32.ba55ae97fe6d26f2fd2a296837570536->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf2/eb76ae13368e4a23855b5680af181f32-bottom, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/cf2/1747520752eb459e951b7491aa7c62a3.ba55ae97fe6d26f2fd2a296837570536->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf2/1747520752eb459e951b7491aa7c62a3-bottom, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/cf2/c332f2f6f5a44943a5e8c97725d45512.ba55ae97fe6d26f2fd2a296837570536->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf2/c332f2f6f5a44943a5e8c97725d45512-bottom, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/cf2/c845912c71644277a0aed468d0621d59.ba55ae97fe6d26f2fd2a296837570536->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf2/c845912c71644277a0aed468d0621d59-bottom, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/cf2/14e73a6c8cd44ed2ab40f783dad882ca.ba55ae97fe6d26f2fd2a296837570536->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf2/14e73a6c8cd44ed2ab40f783dad882ca-bottom, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/cf2/c4912c4098564070a195e3fd918ecec2.ba55ae97fe6d26f2fd2a296837570536->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf2/c4912c4098564070a195e3fd918ecec2-bottom, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/cf2/34f3b8d7d8774320929c1c1bbcc1f132.ba55ae97fe6d26f2fd2a296837570536->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf2/34f3b8d7d8774320929c1c1bbcc1f132-bottom] into tmpdir=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/.tmp, totalSize=34.9 K 2023-05-10 04:55:21,545 DEBUG [RS:0;jenkins-hbase4:36665-shortCompactions-0] compactions.Compactor(207): Compacting 17ee8f61ba9d4f678887f8a52b767dbb.ba55ae97fe6d26f2fd2a296837570536, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1683694516947 2023-05-10 04:55:21,545 DEBUG [RS:0;jenkins-hbase4:36665-longCompactions-0] compactions.Compactor(207): Compacting eb76ae13368e4a23855b5680af181f32.ba55ae97fe6d26f2fd2a296837570536, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1683694516947 2023-05-10 04:55:21,546 DEBUG [RS:0;jenkins-hbase4:36665-longCompactions-0] compactions.Compactor(207): Compacting 1747520752eb459e951b7491aa7c62a3.ba55ae97fe6d26f2fd2a296837570536, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1683694517117 2023-05-10 04:55:21,546 DEBUG [RS:0;jenkins-hbase4:36665-shortCompactions-0] compactions.Compactor(207): Compacting 3488050453f04078bc1365497a2e3f88.ba55ae97fe6d26f2fd2a296837570536, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1683694517117 2023-05-10 04:55:21,546 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2361): Finished post open deploy task for testRecoveryAndDoubleExecution,,1683694518241.48042d1ae86f00fdead71e71ad8292a9. 2023-05-10 04:55:21,546 DEBUG [RS:0;jenkins-hbase4:36665-longCompactions-0] compactions.Compactor(207): Compacting c332f2f6f5a44943a5e8c97725d45512.ba55ae97fe6d26f2fd2a296837570536, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1683694517295 2023-05-10 04:55:21,546 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testRecoveryAndDoubleExecution,,1683694518241.48042d1ae86f00fdead71e71ad8292a9. 2023-05-10 04:55:21,547 DEBUG [RS:0;jenkins-hbase4:36665-shortCompactions-0] compactions.Compactor(207): Compacting c2d599f271fb497b8b272a3bfd391f6e.ba55ae97fe6d26f2fd2a296837570536, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1683694517295 2023-05-10 04:55:21,547 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testRecoveryAndDoubleExecution,41,1683694518241.1b26b3dca87c03702b1dc11207d7b72a. 2023-05-10 04:55:21,547 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=115 updating hbase:meta row=48042d1ae86f00fdead71e71ad8292a9, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:21,547 DEBUG [RS:0;jenkins-hbase4:36665-longCompactions-0] compactions.Compactor(207): Compacting c845912c71644277a0aed468d0621d59.ba55ae97fe6d26f2fd2a296837570536, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1683694517470 2023-05-10 04:55:21,547 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1b26b3dca87c03702b1dc11207d7b72a, NAME => 'testRecoveryAndDoubleExecution,41,1683694518241.1b26b3dca87c03702b1dc11207d7b72a.', STARTKEY => '41', ENDKEY => ''} 2023-05-10 04:55:21,547 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRecoveryAndDoubleExecution,,1683694518241.48042d1ae86f00fdead71e71ad8292a9.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1683694521547"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1683694521547"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1683694521547"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683694521547"}]},"ts":"1683694521547"} 2023-05-10 04:55:21,547 DEBUG [RS:0;jenkins-hbase4:36665-shortCompactions-0] compactions.Compactor(207): Compacting bd5a17faa7564deb857bb9ec28719ee6.ba55ae97fe6d26f2fd2a296837570536, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1683694517470 2023-05-10 04:55:21,547 DEBUG [RS:0;jenkins-hbase4:36665-longCompactions-0] compactions.Compactor(207): Compacting 14e73a6c8cd44ed2ab40f783dad882ca.ba55ae97fe6d26f2fd2a296837570536, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1683694517641 2023-05-10 04:55:21,547 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-05-10 04:55:21,547 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRecoveryAndDoubleExecution 1b26b3dca87c03702b1dc11207d7b72a 2023-05-10 04:55:21,547 DEBUG [RS:0;jenkins-hbase4:36665-shortCompactions-0] compactions.Compactor(207): Compacting b3e182af00c84ddb81eb7331183b802e.ba55ae97fe6d26f2fd2a296837570536, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1683694517641 2023-05-10 04:55:21,547 DEBUG [RS:0;jenkins-hbase4:36665-longCompactions-0] compactions.Compactor(207): Compacting c4912c4098564070a195e3fd918ecec2.ba55ae97fe6d26f2fd2a296837570536, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1683694517811 2023-05-10 04:55:21,547 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testRecoveryAndDoubleExecution,41,1683694518241.1b26b3dca87c03702b1dc11207d7b72a.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-10 04:55:21,548 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 1b26b3dca87c03702b1dc11207d7b72a 2023-05-10 04:55:21,548 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 1b26b3dca87c03702b1dc11207d7b72a 2023-05-10 04:55:21,548 DEBUG [RS:0;jenkins-hbase4:36665-shortCompactions-0] compactions.Compactor(207): Compacting aa810537cf12426fa1bcdd14225b68ee.ba55ae97fe6d26f2fd2a296837570536, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1683694517811 2023-05-10 04:55:21,548 DEBUG [RS:0;jenkins-hbase4:36665-longCompactions-0] compactions.Compactor(207): Compacting 34f3b8d7d8774320929c1c1bbcc1f132.ba55ae97fe6d26f2fd2a296837570536, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1683694517983 2023-05-10 04:55:21,549 DEBUG [RS:0;jenkins-hbase4:36665-shortCompactions-0] compactions.Compactor(207): Compacting 4e150d9797e44e02a343bd11ff7b3dc3.ba55ae97fe6d26f2fd2a296837570536, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1683694517983 2023-05-10 04:55:21,549 INFO [StoreOpener-1b26b3dca87c03702b1dc11207d7b72a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 1b26b3dca87c03702b1dc11207d7b72a 2023-05-10 04:55:21,550 DEBUG [StoreOpener-1b26b3dca87c03702b1dc11207d7b72a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/cf1 2023-05-10 04:55:21,550 DEBUG [StoreOpener-1b26b3dca87c03702b1dc11207d7b72a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/cf1 2023-05-10 04:55:21,551 INFO [StoreOpener-1b26b3dca87c03702b1dc11207d7b72a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1b26b3dca87c03702b1dc11207d7b72a columnFamilyName cf1 2023-05-10 04:55:21,552 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=117, resume processing ppid=115 2023-05-10 04:55:21,552 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=117, ppid=115, state=SUCCESS; OpenRegionProcedure 48042d1ae86f00fdead71e71ad8292a9, server=jenkins-hbase4.apache.org,36665,1683694419452 in 235 msec 2023-05-10 04:55:21,553 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=115, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=48042d1ae86f00fdead71e71ad8292a9, ASSIGN in 394 msec 2023-05-10 04:55:21,563 INFO [RS:0;jenkins-hbase4:36665-longCompactions-0] throttle.PressureAwareThroughputController(145): 48042d1ae86f00fdead71e71ad8292a9#cf2#compaction#106 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-05-10 04:55:21,570 DEBUG [StoreOpener-1b26b3dca87c03702b1dc11207d7b72a-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/cf1/17ee8f61ba9d4f678887f8a52b767dbb.ba55ae97fe6d26f2fd2a296837570536->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf1/17ee8f61ba9d4f678887f8a52b767dbb-top 2023-05-10 04:55:21,572 INFO [RS:0;jenkins-hbase4:36665-shortCompactions-0] throttle.PressureAwareThroughputController(145): 48042d1ae86f00fdead71e71ad8292a9#cf1#compaction#107 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-05-10 04:55:21,574 DEBUG [StoreOpener-1b26b3dca87c03702b1dc11207d7b72a-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/cf1/3488050453f04078bc1365497a2e3f88.ba55ae97fe6d26f2fd2a296837570536->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf1/3488050453f04078bc1365497a2e3f88-top 2023-05-10 04:55:21,577 DEBUG [StoreOpener-1b26b3dca87c03702b1dc11207d7b72a-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/cf1/4e150d9797e44e02a343bd11ff7b3dc3.ba55ae97fe6d26f2fd2a296837570536->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf1/4e150d9797e44e02a343bd11ff7b3dc3-top 2023-05-10 04:55:21,577 DEBUG [RS:0;jenkins-hbase4:36665-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/.tmp/cf2/a804054f1e484a71ac9718acf12a6a3c as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/cf2/a804054f1e484a71ac9718acf12a6a3c 2023-05-10 04:55:21,581 DEBUG [StoreOpener-1b26b3dca87c03702b1dc11207d7b72a-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/cf1/aa810537cf12426fa1bcdd14225b68ee.ba55ae97fe6d26f2fd2a296837570536->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf1/aa810537cf12426fa1bcdd14225b68ee-top 2023-05-10 04:55:21,582 INFO [RS:0;jenkins-hbase4:36665-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 48042d1ae86f00fdead71e71ad8292a9/cf2 of 48042d1ae86f00fdead71e71ad8292a9 into a804054f1e484a71ac9718acf12a6a3c(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-05-10 04:55:21,582 DEBUG [RS:0;jenkins-hbase4:36665-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 48042d1ae86f00fdead71e71ad8292a9: 2023-05-10 04:55:21,582 INFO [RS:0;jenkins-hbase4:36665-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,,1683694518241.48042d1ae86f00fdead71e71ad8292a9., storeName=48042d1ae86f00fdead71e71ad8292a9/cf2, priority=9, startTime=1683694521543; duration=0sec 2023-05-10 04:55:21,582 DEBUG [RS:0;jenkins-hbase4:36665-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-10 04:55:21,583 DEBUG [StoreOpener-1b26b3dca87c03702b1dc11207d7b72a-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/cf1/b3e182af00c84ddb81eb7331183b802e.ba55ae97fe6d26f2fd2a296837570536->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf1/b3e182af00c84ddb81eb7331183b802e-top 2023-05-10 04:55:21,586 DEBUG [StoreOpener-1b26b3dca87c03702b1dc11207d7b72a-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/cf1/bd5a17faa7564deb857bb9ec28719ee6.ba55ae97fe6d26f2fd2a296837570536->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf1/bd5a17faa7564deb857bb9ec28719ee6-top 2023-05-10 04:55:21,589 DEBUG [StoreOpener-1b26b3dca87c03702b1dc11207d7b72a-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/cf1/c2d599f271fb497b8b272a3bfd391f6e.ba55ae97fe6d26f2fd2a296837570536->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf1/c2d599f271fb497b8b272a3bfd391f6e-top 2023-05-10 04:55:21,590 INFO [StoreOpener-1b26b3dca87c03702b1dc11207d7b72a-1] regionserver.HStore(310): Store=1b26b3dca87c03702b1dc11207d7b72a/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-10 04:55:21,590 INFO [StoreOpener-1b26b3dca87c03702b1dc11207d7b72a-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 1b26b3dca87c03702b1dc11207d7b72a 2023-05-10 04:55:21,591 DEBUG [StoreOpener-1b26b3dca87c03702b1dc11207d7b72a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/cf2 2023-05-10 04:55:21,591 DEBUG [StoreOpener-1b26b3dca87c03702b1dc11207d7b72a-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/cf2 2023-05-10 04:55:21,591 INFO [StoreOpener-1b26b3dca87c03702b1dc11207d7b72a-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1b26b3dca87c03702b1dc11207d7b72a columnFamilyName cf2 2023-05-10 04:55:21,593 DEBUG [RS:0;jenkins-hbase4:36665-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/.tmp/cf1/e8d699bc79c24348a272b5b24f3d26d2 as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/cf1/e8d699bc79c24348a272b5b24f3d26d2 2023-05-10 04:55:21,598 INFO [RS:0;jenkins-hbase4:36665-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 48042d1ae86f00fdead71e71ad8292a9/cf1 of 48042d1ae86f00fdead71e71ad8292a9 into e8d699bc79c24348a272b5b24f3d26d2(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-05-10 04:55:21,598 DEBUG [RS:0;jenkins-hbase4:36665-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 48042d1ae86f00fdead71e71ad8292a9: 2023-05-10 04:55:21,598 INFO [RS:0;jenkins-hbase4:36665-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,,1683694518241.48042d1ae86f00fdead71e71ad8292a9., storeName=48042d1ae86f00fdead71e71ad8292a9/cf1, priority=9, startTime=1683694521542; duration=0sec 2023-05-10 04:55:21,598 DEBUG [RS:0;jenkins-hbase4:36665-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-10 04:55:21,603 DEBUG [StoreOpener-1b26b3dca87c03702b1dc11207d7b72a-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/cf2/14e73a6c8cd44ed2ab40f783dad882ca.ba55ae97fe6d26f2fd2a296837570536->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf2/14e73a6c8cd44ed2ab40f783dad882ca-top 2023-05-10 04:55:21,606 DEBUG [StoreOpener-1b26b3dca87c03702b1dc11207d7b72a-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/cf2/1747520752eb459e951b7491aa7c62a3.ba55ae97fe6d26f2fd2a296837570536->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf2/1747520752eb459e951b7491aa7c62a3-top 2023-05-10 04:55:21,608 DEBUG [StoreOpener-1b26b3dca87c03702b1dc11207d7b72a-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/cf2/34f3b8d7d8774320929c1c1bbcc1f132.ba55ae97fe6d26f2fd2a296837570536->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf2/34f3b8d7d8774320929c1c1bbcc1f132-top 2023-05-10 04:55:21,611 DEBUG [StoreOpener-1b26b3dca87c03702b1dc11207d7b72a-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/cf2/c332f2f6f5a44943a5e8c97725d45512.ba55ae97fe6d26f2fd2a296837570536->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf2/c332f2f6f5a44943a5e8c97725d45512-top 2023-05-10 04:55:21,614 DEBUG [StoreOpener-1b26b3dca87c03702b1dc11207d7b72a-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/cf2/c4912c4098564070a195e3fd918ecec2.ba55ae97fe6d26f2fd2a296837570536->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf2/c4912c4098564070a195e3fd918ecec2-top 2023-05-10 04:55:21,617 DEBUG [StoreOpener-1b26b3dca87c03702b1dc11207d7b72a-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/cf2/c845912c71644277a0aed468d0621d59.ba55ae97fe6d26f2fd2a296837570536->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf2/c845912c71644277a0aed468d0621d59-top 2023-05-10 04:55:21,619 DEBUG [StoreOpener-1b26b3dca87c03702b1dc11207d7b72a-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/cf2/eb76ae13368e4a23855b5680af181f32.ba55ae97fe6d26f2fd2a296837570536->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf2/eb76ae13368e4a23855b5680af181f32-top 2023-05-10 04:55:21,619 INFO [StoreOpener-1b26b3dca87c03702b1dc11207d7b72a-1] regionserver.HStore(310): Store=1b26b3dca87c03702b1dc11207d7b72a/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-10 04:55:21,620 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a 2023-05-10 04:55:21,621 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a 2023-05-10 04:55:21,623 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRecoveryAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-05-10 04:55:21,624 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 1b26b3dca87c03702b1dc11207d7b72a 2023-05-10 04:55:21,624 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 1b26b3dca87c03702b1dc11207d7b72a; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67336688, jitterRate=0.0033948421478271484}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-05-10 04:55:21,624 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 1b26b3dca87c03702b1dc11207d7b72a: 2023-05-10 04:55:21,625 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2334): Post open deploy tasks for testRecoveryAndDoubleExecution,41,1683694518241.1b26b3dca87c03702b1dc11207d7b72a., pid=118, masterSystemTime=1683694521466 2023-05-10 04:55:21,625 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-10 04:55:21,626 DEBUG [RS:0;jenkins-hbase4:36665-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-05-10 04:55:21,626 DEBUG [RS:0;jenkins-hbase4:36665-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-05-10 04:55:21,626 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-05-10 04:55:21,626 INFO [RS:0;jenkins-hbase4:36665-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,41,1683694518241.1b26b3dca87c03702b1dc11207d7b72a. 2023-05-10 04:55:21,626 INFO [RS:0;jenkins-hbase4:36665-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,41,1683694518241.1b26b3dca87c03702b1dc11207d7b72a. 2023-05-10 04:55:21,626 DEBUG [RS:0;jenkins-hbase4:36665-longCompactions-0] regionserver.HStore(1912): 1b26b3dca87c03702b1dc11207d7b72a/cf2 is initiating minor compaction (all files) 2023-05-10 04:55:21,626 DEBUG [RS:0;jenkins-hbase4:36665-shortCompactions-0] regionserver.HStore(1912): 1b26b3dca87c03702b1dc11207d7b72a/cf1 is initiating minor compaction (all files) 2023-05-10 04:55:21,626 INFO [RS:0;jenkins-hbase4:36665-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 1b26b3dca87c03702b1dc11207d7b72a/cf2 in testRecoveryAndDoubleExecution,41,1683694518241.1b26b3dca87c03702b1dc11207d7b72a. 2023-05-10 04:55:21,626 INFO [RS:0;jenkins-hbase4:36665-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 1b26b3dca87c03702b1dc11207d7b72a/cf1 in testRecoveryAndDoubleExecution,41,1683694518241.1b26b3dca87c03702b1dc11207d7b72a. 2023-05-10 04:55:21,627 INFO [RS:0;jenkins-hbase4:36665-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/cf2/eb76ae13368e4a23855b5680af181f32.ba55ae97fe6d26f2fd2a296837570536->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf2/eb76ae13368e4a23855b5680af181f32-top, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/cf2/1747520752eb459e951b7491aa7c62a3.ba55ae97fe6d26f2fd2a296837570536->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf2/1747520752eb459e951b7491aa7c62a3-top, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/cf2/c332f2f6f5a44943a5e8c97725d45512.ba55ae97fe6d26f2fd2a296837570536->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf2/c332f2f6f5a44943a5e8c97725d45512-top, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/cf2/c845912c71644277a0aed468d0621d59.ba55ae97fe6d26f2fd2a296837570536->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf2/c845912c71644277a0aed468d0621d59-top, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/cf2/14e73a6c8cd44ed2ab40f783dad882ca.ba55ae97fe6d26f2fd2a296837570536->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf2/14e73a6c8cd44ed2ab40f783dad882ca-top, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/cf2/c4912c4098564070a195e3fd918ecec2.ba55ae97fe6d26f2fd2a296837570536->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf2/c4912c4098564070a195e3fd918ecec2-top, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/cf2/34f3b8d7d8774320929c1c1bbcc1f132.ba55ae97fe6d26f2fd2a296837570536->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf2/34f3b8d7d8774320929c1c1bbcc1f132-top] into tmpdir=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/.tmp, totalSize=34.9 K 2023-05-10 04:55:21,627 INFO [RS:0;jenkins-hbase4:36665-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/cf1/17ee8f61ba9d4f678887f8a52b767dbb.ba55ae97fe6d26f2fd2a296837570536->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf1/17ee8f61ba9d4f678887f8a52b767dbb-top, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/cf1/3488050453f04078bc1365497a2e3f88.ba55ae97fe6d26f2fd2a296837570536->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf1/3488050453f04078bc1365497a2e3f88-top, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/cf1/c2d599f271fb497b8b272a3bfd391f6e.ba55ae97fe6d26f2fd2a296837570536->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf1/c2d599f271fb497b8b272a3bfd391f6e-top, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/cf1/bd5a17faa7564deb857bb9ec28719ee6.ba55ae97fe6d26f2fd2a296837570536->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf1/bd5a17faa7564deb857bb9ec28719ee6-top, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/cf1/b3e182af00c84ddb81eb7331183b802e.ba55ae97fe6d26f2fd2a296837570536->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf1/b3e182af00c84ddb81eb7331183b802e-top, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/cf1/aa810537cf12426fa1bcdd14225b68ee.ba55ae97fe6d26f2fd2a296837570536->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf1/aa810537cf12426fa1bcdd14225b68ee-top, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/cf1/4e150d9797e44e02a343bd11ff7b3dc3.ba55ae97fe6d26f2fd2a296837570536->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf1/4e150d9797e44e02a343bd11ff7b3dc3-top] into tmpdir=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/.tmp, totalSize=34.9 K 2023-05-10 04:55:21,627 DEBUG [RS:0;jenkins-hbase4:36665-longCompactions-0] compactions.Compactor(207): Compacting eb76ae13368e4a23855b5680af181f32.ba55ae97fe6d26f2fd2a296837570536, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1683694516947 2023-05-10 04:55:21,628 DEBUG [RS:0;jenkins-hbase4:36665-shortCompactions-0] compactions.Compactor(207): Compacting 17ee8f61ba9d4f678887f8a52b767dbb.ba55ae97fe6d26f2fd2a296837570536, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1683694516947 2023-05-10 04:55:21,628 DEBUG [RS:0;jenkins-hbase4:36665-longCompactions-0] compactions.Compactor(207): Compacting 1747520752eb459e951b7491aa7c62a3.ba55ae97fe6d26f2fd2a296837570536, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1683694517117 2023-05-10 04:55:21,628 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2361): Finished post open deploy task for testRecoveryAndDoubleExecution,41,1683694518241.1b26b3dca87c03702b1dc11207d7b72a. 2023-05-10 04:55:21,628 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testRecoveryAndDoubleExecution,41,1683694518241.1b26b3dca87c03702b1dc11207d7b72a. 2023-05-10 04:55:21,628 DEBUG [RS:0;jenkins-hbase4:36665-shortCompactions-0] compactions.Compactor(207): Compacting 3488050453f04078bc1365497a2e3f88.ba55ae97fe6d26f2fd2a296837570536, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1683694517117 2023-05-10 04:55:21,628 DEBUG [RS:0;jenkins-hbase4:36665-longCompactions-0] compactions.Compactor(207): Compacting c332f2f6f5a44943a5e8c97725d45512.ba55ae97fe6d26f2fd2a296837570536, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1683694517295 2023-05-10 04:55:21,628 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=116 updating hbase:meta row=1b26b3dca87c03702b1dc11207d7b72a, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:21,628 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRecoveryAndDoubleExecution,41,1683694518241.1b26b3dca87c03702b1dc11207d7b72a.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1683694521628"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1683694521628"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1683694521628"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683694521628"}]},"ts":"1683694521628"} 2023-05-10 04:55:21,628 DEBUG [RS:0;jenkins-hbase4:36665-shortCompactions-0] compactions.Compactor(207): Compacting c2d599f271fb497b8b272a3bfd391f6e.ba55ae97fe6d26f2fd2a296837570536, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1683694517295 2023-05-10 04:55:21,629 DEBUG [RS:0;jenkins-hbase4:36665-longCompactions-0] compactions.Compactor(207): Compacting c845912c71644277a0aed468d0621d59.ba55ae97fe6d26f2fd2a296837570536, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1683694517470 2023-05-10 04:55:21,629 DEBUG [RS:0;jenkins-hbase4:36665-shortCompactions-0] compactions.Compactor(207): Compacting bd5a17faa7564deb857bb9ec28719ee6.ba55ae97fe6d26f2fd2a296837570536, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1683694517470 2023-05-10 04:55:21,629 DEBUG [RS:0;jenkins-hbase4:36665-longCompactions-0] compactions.Compactor(207): Compacting 14e73a6c8cd44ed2ab40f783dad882ca.ba55ae97fe6d26f2fd2a296837570536, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1683694517641 2023-05-10 04:55:21,629 DEBUG [RS:0;jenkins-hbase4:36665-shortCompactions-0] compactions.Compactor(207): Compacting b3e182af00c84ddb81eb7331183b802e.ba55ae97fe6d26f2fd2a296837570536, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1683694517641 2023-05-10 04:55:21,630 DEBUG [RS:0;jenkins-hbase4:36665-longCompactions-0] compactions.Compactor(207): Compacting c4912c4098564070a195e3fd918ecec2.ba55ae97fe6d26f2fd2a296837570536, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1683694517811 2023-05-10 04:55:21,630 DEBUG [RS:0;jenkins-hbase4:36665-shortCompactions-0] compactions.Compactor(207): Compacting aa810537cf12426fa1bcdd14225b68ee.ba55ae97fe6d26f2fd2a296837570536, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1683694517811 2023-05-10 04:55:21,630 DEBUG [RS:0;jenkins-hbase4:36665-longCompactions-0] compactions.Compactor(207): Compacting 34f3b8d7d8774320929c1c1bbcc1f132.ba55ae97fe6d26f2fd2a296837570536, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1683694517983 2023-05-10 04:55:21,630 DEBUG [RS:0;jenkins-hbase4:36665-shortCompactions-0] compactions.Compactor(207): Compacting 4e150d9797e44e02a343bd11ff7b3dc3.ba55ae97fe6d26f2fd2a296837570536, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1683694517983 2023-05-10 04:55:21,631 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=118, resume processing ppid=116 2023-05-10 04:55:21,631 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=118, ppid=116, state=SUCCESS; OpenRegionProcedure 1b26b3dca87c03702b1dc11207d7b72a, server=jenkins-hbase4.apache.org,36665,1683694419452 in 312 msec 2023-05-10 04:55:21,633 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=116, resume processing ppid=112 2023-05-10 04:55:21,633 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=116, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=1b26b3dca87c03702b1dc11207d7b72a, ASSIGN in 473 msec 2023-05-10 04:55:21,633 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-05-10 04:55:21,633 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=SUCCESS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ba55ae97fe6d26f2fd2a296837570536, daughterA=48042d1ae86f00fdead71e71ad8292a9, daughterB=1b26b3dca87c03702b1dc11207d7b72a 2023-05-10 04:55:21,633 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-05-10 04:55:21,633 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=SUCCESS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ba55ae97fe6d26f2fd2a296837570536, daughterA=48042d1ae86f00fdead71e71ad8292a9, daughterB=1b26b3dca87c03702b1dc11207d7b72a at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-05-10 04:55:21,639 INFO [RS:0;jenkins-hbase4:36665-longCompactions-0] throttle.PressureAwareThroughputController(145): 1b26b3dca87c03702b1dc11207d7b72a#cf2#compaction#108 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-05-10 04:55:21,640 INFO [RS:0;jenkins-hbase4:36665-shortCompactions-0] throttle.PressureAwareThroughputController(145): 1b26b3dca87c03702b1dc11207d7b72a#cf1#compaction#109 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-05-10 04:55:21,657 DEBUG [RS:0;jenkins-hbase4:36665-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/.tmp/cf2/53c2352e58614b309a0a33f58376937f as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/cf2/53c2352e58614b309a0a33f58376937f 2023-05-10 04:55:21,658 INFO [Listener at localhost/45007] procedure.MasterProcedureTestingUtility(454): Restart 10 exec state=pid=112, state=SUCCESS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ba55ae97fe6d26f2fd2a296837570536, daughterA=48042d1ae86f00fdead71e71ad8292a9, daughterB=1b26b3dca87c03702b1dc11207d7b72a 2023-05-10 04:55:21,658 INFO [Listener at localhost/45007] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-05-10 04:55:21,658 INFO [Listener at localhost/45007] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-10 04:55:21,658 INFO [Listener at localhost/45007] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-10 04:55:21,659 ERROR [Listener at localhost/45007] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-05-10 04:55:21,659 INFO [Listener at localhost/45007] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-05-10 04:55:21,659 INFO [Listener at localhost/45007] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-05-10 04:55:21,659 INFO [Listener at localhost/45007] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-05-10 04:55:21,659 INFO [Listener at localhost/45007] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-05-10 04:55:21,660 INFO [Listener at localhost/45007] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-05-10 04:55:21,663 DEBUG [Listener at localhost/45007] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10, 11, 12, 13] 2023-05-10 04:55:21,663 DEBUG [Listener at localhost/45007] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-05-10 04:55:21,663 DEBUG [Listener at localhost/45007] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-05-10 04:55:21,663 DEBUG [Listener at localhost/45007] store.ProcedureTree(114): Procedure Procedure(pid=115, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[15, 16, 21] 2023-05-10 04:55:21,663 DEBUG [Listener at localhost/45007] store.ProcedureTree(114): Procedure Procedure(pid=117, ppid=115, class=org.apache.hadoop.hbase.master.assignment.OpenRegionProcedure) stack ids=[17, 20] 2023-05-10 04:55:21,663 DEBUG [Listener at localhost/45007] store.ProcedureTree(114): Procedure Procedure(pid=116, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[14, 18, 23] 2023-05-10 04:55:21,663 DEBUG [Listener at localhost/45007] store.ProcedureTree(114): Procedure Procedure(pid=118, ppid=116, class=org.apache.hadoop.hbase.master.assignment.OpenRegionProcedure) stack ids=[19, 22] 2023-05-10 04:55:21,664 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-05-10 04:55:21,664 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-05-10 04:55:21,664 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-05-10 04:55:21,664 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-05-10 04:55:21,665 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:55:21,665 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=93c2cc329b1fe1893cfbc28c56960522, daughterA=19040b6538a6bdeed7c833cfb44614b6, daughterB=054f69c06f126aad97b290410d5d6188 2023-05-10 04:55:21,665 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:55:21,665 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-05-10 04:55:21,666 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-05-10 04:55:21,666 INFO [RS:0;jenkins-hbase4:36665-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 1b26b3dca87c03702b1dc11207d7b72a/cf2 of 1b26b3dca87c03702b1dc11207d7b72a into 53c2352e58614b309a0a33f58376937f(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-05-10 04:55:21,666 DEBUG [RS:0;jenkins-hbase4:36665-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 1b26b3dca87c03702b1dc11207d7b72a: 2023-05-10 04:55:21,666 INFO [RS:0;jenkins-hbase4:36665-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,41,1683694518241.1b26b3dca87c03702b1dc11207d7b72a., storeName=1b26b3dca87c03702b1dc11207d7b72a/cf2, priority=9, startTime=1683694521626; duration=0sec 2023-05-10 04:55:21,666 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6c70954228a0bbe97dfdc76861367c66, daughterA=75e2592cb2ac08bd9f617bb9d0a0c896, daughterB=b90fd344aff14f0728a80c3d09539e54 2023-05-10 04:55:21,666 DEBUG [RS:0;jenkins-hbase4:36665-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-10 04:55:21,666 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-05-10 04:55:21,666 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-05-10 04:55:21,667 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-05-10 04:55:21,667 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=4dcb62249031b560a1bc5d03e43ac855, daughterA=0bdfce2d16e0db079723da4113f9942f, daughterB=163c5ab21f256d68dd3647b5b3519220 2023-05-10 04:55:21,667 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=4dcb62249031b560a1bc5d03e43ac855, ASSIGN 2023-05-10 04:55:21,667 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-05-10 04:55:21,667 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-05-10 04:55:21,668 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-10 04:55:21,668 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=a7ae61c05a799883e33cfd9dfad56fa1, daughterA=944df92d059326a46982ff7979f8a940, daughterB=fa1eb74e4c5521990fb9118cc6dfd407 2023-05-10 04:55:21,668 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-10 04:55:21,668 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-05-10 04:55:21,669 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:55:21,669 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=e0a753dfc7b1edf68920ae1543edc38c, daughterA=08557250442eebc7c86235cdf682b1a9, daughterB=e1dcbf091d2db6dbef8d701eb3d89914 2023-05-10 04:55:21,669 DEBUG [RS:0;jenkins-hbase4:36665-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/.tmp/cf1/e7f43908ebe941f5b96351fbdfd3db7d as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/cf1/e7f43908ebe941f5b96351fbdfd3db7d 2023-05-10 04:55:21,669 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:55:21,669 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-05-10 04:55:21,669 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-10 04:55:21,670 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-10 04:55:21,670 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase4.apache.org,37995,1683694419499, table=testRollbackForSplitTableRegionWithReplica, region=bddd3bfc356670bc9cd9492c47d95a01 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=d2b34babccb97d3c1ac066d9e90ae654, daughterA=0141d5003cd2aac0ddef526f1a98709b, daughterB=b919ef6ce9c770ae26c6c5ee0d2ac459 2023-05-10 04:55:21,670 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-10 04:55:21,670 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-05-10 04:55:21,671 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-05-10 04:55:21,671 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-05-10 04:55:21,671 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-05-10 04:55:21,671 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-05-10 04:55:21,673 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ba55ae97fe6d26f2fd2a296837570536, daughterA=48042d1ae86f00fdead71e71ad8292a9, daughterB=1b26b3dca87c03702b1dc11207d7b72a 2023-05-10 04:55:21,673 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=ba55ae97fe6d26f2fd2a296837570536, UNASSIGN 2023-05-10 04:55:21,673 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure ba55ae97fe6d26f2fd2a296837570536, server=jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:21,673 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(468): Loading pid=115, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=48042d1ae86f00fdead71e71ad8292a9, ASSIGN 2023-05-10 04:55:21,673 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(468): Loading pid=116, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=1b26b3dca87c03702b1dc11207d7b72a, ASSIGN 2023-05-10 04:55:21,673 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(468): Loading pid=117, ppid=115, state=SUCCESS; OpenRegionProcedure 48042d1ae86f00fdead71e71ad8292a9, server=jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:21,673 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(468): Loading pid=118, ppid=116, state=SUCCESS; OpenRegionProcedure 1b26b3dca87c03702b1dc11207d7b72a, server=jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:21,673 DEBUG [Listener at localhost/45007] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ba55ae97fe6d26f2fd2a296837570536, daughterA=48042d1ae86f00fdead71e71ad8292a9, daughterB=1b26b3dca87c03702b1dc11207d7b72a held the lock before restarting, call acquireLock to restore it. 2023-05-10 04:55:21,674 INFO [Listener at localhost/45007] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ba55ae97fe6d26f2fd2a296837570536, daughterA=48042d1ae86f00fdead71e71ad8292a9, daughterB=1b26b3dca87c03702b1dc11207d7b72a 2023-05-10 04:55:21,674 INFO [Listener at localhost/45007] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ba55ae97fe6d26f2fd2a296837570536, daughterA=48042d1ae86f00fdead71e71ad8292a9, daughterB=1b26b3dca87c03702b1dc11207d7b72a 2023-05-10 04:55:21,674 INFO [Listener at localhost/45007] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ba55ae97fe6d26f2fd2a296837570536, daughterA=48042d1ae86f00fdead71e71ad8292a9, daughterB=1b26b3dca87c03702b1dc11207d7b72a 2023-05-10 04:55:21,674 DEBUG [Listener at localhost/45007] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=ba55ae97fe6d26f2fd2a296837570536, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-05-10 04:55:21,674 DEBUG [Listener at localhost/45007] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure ba55ae97fe6d26f2fd2a296837570536, server=jenkins-hbase4.apache.org,36665,1683694419452 didn't hold the lock before restarting, skip acquiring lock. 2023-05-10 04:55:21,674 DEBUG [Listener at localhost/45007] procedure2.Procedure(946): pid=115, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=48042d1ae86f00fdead71e71ad8292a9, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-05-10 04:55:21,674 DEBUG [Listener at localhost/45007] procedure2.Procedure(946): pid=116, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=1b26b3dca87c03702b1dc11207d7b72a, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-05-10 04:55:21,674 INFO [RS:0;jenkins-hbase4:36665-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 1b26b3dca87c03702b1dc11207d7b72a/cf1 of 1b26b3dca87c03702b1dc11207d7b72a into e7f43908ebe941f5b96351fbdfd3db7d(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-05-10 04:55:21,674 DEBUG [Listener at localhost/45007] procedure2.Procedure(946): pid=117, ppid=115, state=SUCCESS; OpenRegionProcedure 48042d1ae86f00fdead71e71ad8292a9, server=jenkins-hbase4.apache.org,36665,1683694419452 didn't hold the lock before restarting, skip acquiring lock. 2023-05-10 04:55:21,675 DEBUG [RS:0;jenkins-hbase4:36665-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 1b26b3dca87c03702b1dc11207d7b72a: 2023-05-10 04:55:21,675 DEBUG [Listener at localhost/45007] procedure2.Procedure(946): pid=118, ppid=116, state=SUCCESS; OpenRegionProcedure 1b26b3dca87c03702b1dc11207d7b72a, server=jenkins-hbase4.apache.org,36665,1683694419452 didn't hold the lock before restarting, skip acquiring lock. 2023-05-10 04:55:21,675 INFO [RS:0;jenkins-hbase4:36665-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,41,1683694518241.1b26b3dca87c03702b1dc11207d7b72a., storeName=1b26b3dca87c03702b1dc11207d7b72a/cf1, priority=9, startTime=1683694521625; duration=0sec 2023-05-10 04:55:21,675 INFO [Listener at localhost/45007] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 14 msec 2023-05-10 04:55:21,675 DEBUG [RS:0;jenkins-hbase4:36665-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-10 04:55:21,675 DEBUG [Listener at localhost/45007] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-05-10 04:55:21,676 DEBUG [Listener at localhost/45007] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase4.apache.org,37995,1683694419499, table=hbase:meta, region=1588230740 2023-05-10 04:55:21,680 INFO [Listener at localhost/45007] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1683694551680 2023-05-10 04:55:21,682 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ba55ae97fe6d26f2fd2a296837570536, daughterA=48042d1ae86f00fdead71e71ad8292a9, daughterB=1b26b3dca87c03702b1dc11207d7b72a 2023-05-10 04:55:21,682 DEBUG [Listener at localhost/45007] assignment.AssignmentManager(1517): Joining cluster... 2023-05-10 04:55:21,685 DEBUG [Listener at localhost/45007] assignment.RegionStateStore(147): Load hbase:meta entry region=ccfda81dc2f13ffc1429bef67d349028, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36665,1683694419452, regionLocation=jenkins-hbase4.apache.org,36665,1683694419452, openSeqNum=2 2023-05-10 04:55:21,685 DEBUG [Listener at localhost/45007] assignment.RegionStateStore(147): Load hbase:meta entry region=ba55ae97fe6d26f2fd2a296837570536, regionState=CLOSED, lastHost=jenkins-hbase4.apache.org,36665,1683694419452, regionLocation=jenkins-hbase4.apache.org,36665,1683694419452, openSeqNum=2 2023-05-10 04:55:21,685 INFO [Listener at localhost/45007] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase4.apache.org,36665,1683694419452, table=testRecoveryAndDoubleExecution, region=ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:21,685 DEBUG [Listener at localhost/45007] assignment.RegionStateStore(147): Load hbase:meta entry region=48042d1ae86f00fdead71e71ad8292a9, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36665,1683694419452, regionLocation=jenkins-hbase4.apache.org,36665,1683694419452, openSeqNum=86 2023-05-10 04:55:21,686 DEBUG [Listener at localhost/45007] assignment.RegionStateStore(147): Load hbase:meta entry region=1b26b3dca87c03702b1dc11207d7b72a, regionState=OPEN, lastHost=jenkins-hbase4.apache.org,36665,1683694419452, regionLocation=jenkins-hbase4.apache.org,36665,1683694419452, openSeqNum=86 2023-05-10 04:55:21,686 INFO [Listener at localhost/45007] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-05-10 04:55:21,686 INFO [Listener at localhost/45007] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1683694581686 2023-05-10 04:55:21,686 INFO [Listener at localhost/45007] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1683694641686 2023-05-10 04:55:21,686 INFO [Listener at localhost/45007] assignment.AssignmentManager(1536): Joined the cluster in 4 msec 2023-05-10 04:55:21,686 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-05-10 04:55:21,689 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=112, state=SUCCESS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=ba55ae97fe6d26f2fd2a296837570536, daughterA=48042d1ae86f00fdead71e71ad8292a9, daughterB=1b26b3dca87c03702b1dc11207d7b72a in 3.4450 sec 2023-05-10 04:55:21,937 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36665] ipc.CallRunner(144): callId: 888 service: ClientService methodName: Scan size: 154 connection: 172.31.14.131:54092 deadline: 1683694581937, exception=org.apache.hadoop.hbase.NotServingRegionException: testRecoveryAndDoubleExecution,,1683694516443.ba55ae97fe6d26f2fd2a296837570536. is not online on jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:22,048 DEBUG [Listener at localhost/45007] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-05-10 04:55:22,056 INFO [Listener at localhost/45007] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-05-10 04:55:22,057 DEBUG [Listener at localhost/45007] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-05-10 04:55:22,065 WARN [Listener at localhost/45007] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-05-10 04:55:22,066 INFO [Listener at localhost/45007] client.HBaseAdmin$15(890): Started disable of testRecoveryAndDoubleExecution 2023-05-10 04:55:22,067 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testRecoveryAndDoubleExecution 2023-05-10 04:55:22,067 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] procedure2.ProcedureExecutor(1029): Stored pid=119, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testRecoveryAndDoubleExecution 2023-05-10 04:55:22,070 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(1227): Checking to see if procedure is done pid=119 2023-05-10 04:55:22,070 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683694522070"}]},"ts":"1683694522070"} 2023-05-10 04:55:22,071 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=DISABLING in hbase:meta 2023-05-10 04:55:22,072 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testRecoveryAndDoubleExecution to state=DISABLING 2023-05-10 04:55:22,073 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=120, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=48042d1ae86f00fdead71e71ad8292a9, UNASSIGN}, {pid=121, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=1b26b3dca87c03702b1dc11207d7b72a, UNASSIGN}] 2023-05-10 04:55:22,074 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=121, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=1b26b3dca87c03702b1dc11207d7b72a, UNASSIGN 2023-05-10 04:55:22,074 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=121 updating hbase:meta row=1b26b3dca87c03702b1dc11207d7b72a, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:22,074 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1683694518241.1b26b3dca87c03702b1dc11207d7b72a.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1683694522074"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683694522074"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683694522074"}]},"ts":"1683694522074"} 2023-05-10 04:55:22,075 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=122, ppid=121, state=RUNNABLE; CloseRegionProcedure 1b26b3dca87c03702b1dc11207d7b72a, server=jenkins-hbase4.apache.org,36665,1683694419452}] 2023-05-10 04:55:22,077 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=120, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=48042d1ae86f00fdead71e71ad8292a9, UNASSIGN 2023-05-10 04:55:22,078 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=120 updating hbase:meta row=48042d1ae86f00fdead71e71ad8292a9, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:22,078 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1683694518241.48042d1ae86f00fdead71e71ad8292a9.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1683694522078"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683694522078"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683694522078"}]},"ts":"1683694522078"} 2023-05-10 04:55:22,079 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=123, ppid=120, state=RUNNABLE; CloseRegionProcedure 48042d1ae86f00fdead71e71ad8292a9, server=jenkins-hbase4.apache.org,36665,1683694419452}] 2023-05-10 04:55:22,171 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(1227): Checking to see if procedure is done pid=119 2023-05-10 04:55:22,227 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 48042d1ae86f00fdead71e71ad8292a9 2023-05-10 04:55:22,227 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 48042d1ae86f00fdead71e71ad8292a9, disabling compactions & flushes 2023-05-10 04:55:22,227 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testRecoveryAndDoubleExecution,,1683694518241.48042d1ae86f00fdead71e71ad8292a9. 2023-05-10 04:55:22,227 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,,1683694518241.48042d1ae86f00fdead71e71ad8292a9. 2023-05-10 04:55:22,227 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testRecoveryAndDoubleExecution,,1683694518241.48042d1ae86f00fdead71e71ad8292a9. after waiting 0 ms 2023-05-10 04:55:22,227 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testRecoveryAndDoubleExecution,,1683694518241.48042d1ae86f00fdead71e71ad8292a9. 2023-05-10 04:55:22,228 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1683694518241.48042d1ae86f00fdead71e71ad8292a9.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/cf1/17ee8f61ba9d4f678887f8a52b767dbb.ba55ae97fe6d26f2fd2a296837570536->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf1/17ee8f61ba9d4f678887f8a52b767dbb-bottom, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/cf1/3488050453f04078bc1365497a2e3f88.ba55ae97fe6d26f2fd2a296837570536->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf1/3488050453f04078bc1365497a2e3f88-bottom, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/cf1/c2d599f271fb497b8b272a3bfd391f6e.ba55ae97fe6d26f2fd2a296837570536->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf1/c2d599f271fb497b8b272a3bfd391f6e-bottom, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/cf1/bd5a17faa7564deb857bb9ec28719ee6.ba55ae97fe6d26f2fd2a296837570536->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf1/bd5a17faa7564deb857bb9ec28719ee6-bottom, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/cf1/b3e182af00c84ddb81eb7331183b802e.ba55ae97fe6d26f2fd2a296837570536->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf1/b3e182af00c84ddb81eb7331183b802e-bottom, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/cf1/aa810537cf12426fa1bcdd14225b68ee.ba55ae97fe6d26f2fd2a296837570536->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf1/aa810537cf12426fa1bcdd14225b68ee-bottom, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/cf1/4e150d9797e44e02a343bd11ff7b3dc3.ba55ae97fe6d26f2fd2a296837570536->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf1/4e150d9797e44e02a343bd11ff7b3dc3-bottom] to archive 2023-05-10 04:55:22,229 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1683694518241.48042d1ae86f00fdead71e71ad8292a9.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-05-10 04:55:22,231 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1683694518241.48042d1ae86f00fdead71e71ad8292a9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/cf1/17ee8f61ba9d4f678887f8a52b767dbb.ba55ae97fe6d26f2fd2a296837570536 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/cf1/17ee8f61ba9d4f678887f8a52b767dbb.ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:22,232 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1683694518241.48042d1ae86f00fdead71e71ad8292a9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/cf1/3488050453f04078bc1365497a2e3f88.ba55ae97fe6d26f2fd2a296837570536 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/cf1/3488050453f04078bc1365497a2e3f88.ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:22,233 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1683694518241.48042d1ae86f00fdead71e71ad8292a9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/cf1/c2d599f271fb497b8b272a3bfd391f6e.ba55ae97fe6d26f2fd2a296837570536 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/cf1/c2d599f271fb497b8b272a3bfd391f6e.ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:22,234 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1683694518241.48042d1ae86f00fdead71e71ad8292a9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/cf1/bd5a17faa7564deb857bb9ec28719ee6.ba55ae97fe6d26f2fd2a296837570536 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/cf1/bd5a17faa7564deb857bb9ec28719ee6.ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:22,235 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1683694518241.48042d1ae86f00fdead71e71ad8292a9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/cf1/b3e182af00c84ddb81eb7331183b802e.ba55ae97fe6d26f2fd2a296837570536 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/cf1/b3e182af00c84ddb81eb7331183b802e.ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:22,236 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1683694518241.48042d1ae86f00fdead71e71ad8292a9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/cf1/aa810537cf12426fa1bcdd14225b68ee.ba55ae97fe6d26f2fd2a296837570536 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/cf1/aa810537cf12426fa1bcdd14225b68ee.ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:22,237 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1683694518241.48042d1ae86f00fdead71e71ad8292a9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/cf1/4e150d9797e44e02a343bd11ff7b3dc3.ba55ae97fe6d26f2fd2a296837570536 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/cf1/4e150d9797e44e02a343bd11ff7b3dc3.ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:22,238 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1683694518241.48042d1ae86f00fdead71e71ad8292a9.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/cf2/eb76ae13368e4a23855b5680af181f32.ba55ae97fe6d26f2fd2a296837570536->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf2/eb76ae13368e4a23855b5680af181f32-bottom, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/cf2/1747520752eb459e951b7491aa7c62a3.ba55ae97fe6d26f2fd2a296837570536->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf2/1747520752eb459e951b7491aa7c62a3-bottom, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/cf2/c332f2f6f5a44943a5e8c97725d45512.ba55ae97fe6d26f2fd2a296837570536->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf2/c332f2f6f5a44943a5e8c97725d45512-bottom, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/cf2/c845912c71644277a0aed468d0621d59.ba55ae97fe6d26f2fd2a296837570536->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf2/c845912c71644277a0aed468d0621d59-bottom, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/cf2/14e73a6c8cd44ed2ab40f783dad882ca.ba55ae97fe6d26f2fd2a296837570536->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf2/14e73a6c8cd44ed2ab40f783dad882ca-bottom, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/cf2/c4912c4098564070a195e3fd918ecec2.ba55ae97fe6d26f2fd2a296837570536->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf2/c4912c4098564070a195e3fd918ecec2-bottom, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/cf2/34f3b8d7d8774320929c1c1bbcc1f132.ba55ae97fe6d26f2fd2a296837570536->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf2/34f3b8d7d8774320929c1c1bbcc1f132-bottom] to archive 2023-05-10 04:55:22,239 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1683694518241.48042d1ae86f00fdead71e71ad8292a9.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-05-10 04:55:22,240 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1683694518241.48042d1ae86f00fdead71e71ad8292a9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/cf2/eb76ae13368e4a23855b5680af181f32.ba55ae97fe6d26f2fd2a296837570536 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/cf2/eb76ae13368e4a23855b5680af181f32.ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:22,241 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1683694518241.48042d1ae86f00fdead71e71ad8292a9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/cf2/1747520752eb459e951b7491aa7c62a3.ba55ae97fe6d26f2fd2a296837570536 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/cf2/1747520752eb459e951b7491aa7c62a3.ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:22,242 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1683694518241.48042d1ae86f00fdead71e71ad8292a9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/cf2/c332f2f6f5a44943a5e8c97725d45512.ba55ae97fe6d26f2fd2a296837570536 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/cf2/c332f2f6f5a44943a5e8c97725d45512.ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:22,243 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1683694518241.48042d1ae86f00fdead71e71ad8292a9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/cf2/c845912c71644277a0aed468d0621d59.ba55ae97fe6d26f2fd2a296837570536 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/cf2/c845912c71644277a0aed468d0621d59.ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:22,244 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1683694518241.48042d1ae86f00fdead71e71ad8292a9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/cf2/14e73a6c8cd44ed2ab40f783dad882ca.ba55ae97fe6d26f2fd2a296837570536 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/cf2/14e73a6c8cd44ed2ab40f783dad882ca.ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:22,245 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1683694518241.48042d1ae86f00fdead71e71ad8292a9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/cf2/c4912c4098564070a195e3fd918ecec2.ba55ae97fe6d26f2fd2a296837570536 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/cf2/c4912c4098564070a195e3fd918ecec2.ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:22,246 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1683694518241.48042d1ae86f00fdead71e71ad8292a9.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/cf2/34f3b8d7d8774320929c1c1bbcc1f132.ba55ae97fe6d26f2fd2a296837570536 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/cf2/34f3b8d7d8774320929c1c1bbcc1f132.ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:22,249 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-05-10 04:55:22,250 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-10 04:55:22,250 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testRecoveryAndDoubleExecution,,1683694518241.48042d1ae86f00fdead71e71ad8292a9. 2023-05-10 04:55:22,250 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 48042d1ae86f00fdead71e71ad8292a9: 2023-05-10 04:55:22,252 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 48042d1ae86f00fdead71e71ad8292a9 2023-05-10 04:55:22,252 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 1b26b3dca87c03702b1dc11207d7b72a 2023-05-10 04:55:22,252 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1b26b3dca87c03702b1dc11207d7b72a, disabling compactions & flushes 2023-05-10 04:55:22,252 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testRecoveryAndDoubleExecution,41,1683694518241.1b26b3dca87c03702b1dc11207d7b72a. 2023-05-10 04:55:22,252 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,41,1683694518241.1b26b3dca87c03702b1dc11207d7b72a. 2023-05-10 04:55:22,252 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testRecoveryAndDoubleExecution,41,1683694518241.1b26b3dca87c03702b1dc11207d7b72a. after waiting 0 ms 2023-05-10 04:55:22,252 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testRecoveryAndDoubleExecution,41,1683694518241.1b26b3dca87c03702b1dc11207d7b72a. 2023-05-10 04:55:22,252 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=120 updating hbase:meta row=48042d1ae86f00fdead71e71ad8292a9, regionState=CLOSED 2023-05-10 04:55:22,253 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,,1683694518241.48042d1ae86f00fdead71e71ad8292a9.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1683694522252"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683694522252"}]},"ts":"1683694522252"} 2023-05-10 04:55:22,253 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1683694518241.1b26b3dca87c03702b1dc11207d7b72a.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/cf1/17ee8f61ba9d4f678887f8a52b767dbb.ba55ae97fe6d26f2fd2a296837570536->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf1/17ee8f61ba9d4f678887f8a52b767dbb-top, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/cf1/3488050453f04078bc1365497a2e3f88.ba55ae97fe6d26f2fd2a296837570536->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf1/3488050453f04078bc1365497a2e3f88-top, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/cf1/c2d599f271fb497b8b272a3bfd391f6e.ba55ae97fe6d26f2fd2a296837570536->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf1/c2d599f271fb497b8b272a3bfd391f6e-top, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/cf1/bd5a17faa7564deb857bb9ec28719ee6.ba55ae97fe6d26f2fd2a296837570536->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf1/bd5a17faa7564deb857bb9ec28719ee6-top, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/cf1/b3e182af00c84ddb81eb7331183b802e.ba55ae97fe6d26f2fd2a296837570536->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf1/b3e182af00c84ddb81eb7331183b802e-top, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/cf1/aa810537cf12426fa1bcdd14225b68ee.ba55ae97fe6d26f2fd2a296837570536->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf1/aa810537cf12426fa1bcdd14225b68ee-top, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/cf1/4e150d9797e44e02a343bd11ff7b3dc3.ba55ae97fe6d26f2fd2a296837570536->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf1/4e150d9797e44e02a343bd11ff7b3dc3-top] to archive 2023-05-10 04:55:22,254 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1683694518241.1b26b3dca87c03702b1dc11207d7b72a.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-05-10 04:55:22,255 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=123, resume processing ppid=120 2023-05-10 04:55:22,255 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=123, ppid=120, state=SUCCESS; CloseRegionProcedure 48042d1ae86f00fdead71e71ad8292a9, server=jenkins-hbase4.apache.org,36665,1683694419452 in 175 msec 2023-05-10 04:55:22,256 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1683694518241.1b26b3dca87c03702b1dc11207d7b72a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/cf1/17ee8f61ba9d4f678887f8a52b767dbb.ba55ae97fe6d26f2fd2a296837570536 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/cf1/17ee8f61ba9d4f678887f8a52b767dbb.ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:22,256 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=120, ppid=119, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=48042d1ae86f00fdead71e71ad8292a9, UNASSIGN in 182 msec 2023-05-10 04:55:22,257 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1683694518241.1b26b3dca87c03702b1dc11207d7b72a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/cf1/3488050453f04078bc1365497a2e3f88.ba55ae97fe6d26f2fd2a296837570536 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/cf1/3488050453f04078bc1365497a2e3f88.ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:22,257 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1683694518241.1b26b3dca87c03702b1dc11207d7b72a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/cf1/c2d599f271fb497b8b272a3bfd391f6e.ba55ae97fe6d26f2fd2a296837570536 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/cf1/c2d599f271fb497b8b272a3bfd391f6e.ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:22,258 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1683694518241.1b26b3dca87c03702b1dc11207d7b72a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/cf1/bd5a17faa7564deb857bb9ec28719ee6.ba55ae97fe6d26f2fd2a296837570536 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/cf1/bd5a17faa7564deb857bb9ec28719ee6.ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:22,259 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1683694518241.1b26b3dca87c03702b1dc11207d7b72a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/cf1/b3e182af00c84ddb81eb7331183b802e.ba55ae97fe6d26f2fd2a296837570536 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/cf1/b3e182af00c84ddb81eb7331183b802e.ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:22,260 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1683694518241.1b26b3dca87c03702b1dc11207d7b72a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/cf1/aa810537cf12426fa1bcdd14225b68ee.ba55ae97fe6d26f2fd2a296837570536 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/cf1/aa810537cf12426fa1bcdd14225b68ee.ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:22,261 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1683694518241.1b26b3dca87c03702b1dc11207d7b72a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/cf1/4e150d9797e44e02a343bd11ff7b3dc3.ba55ae97fe6d26f2fd2a296837570536 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/cf1/4e150d9797e44e02a343bd11ff7b3dc3.ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:22,263 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1683694518241.1b26b3dca87c03702b1dc11207d7b72a.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/cf2/eb76ae13368e4a23855b5680af181f32.ba55ae97fe6d26f2fd2a296837570536->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf2/eb76ae13368e4a23855b5680af181f32-top, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/cf2/1747520752eb459e951b7491aa7c62a3.ba55ae97fe6d26f2fd2a296837570536->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf2/1747520752eb459e951b7491aa7c62a3-top, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/cf2/c332f2f6f5a44943a5e8c97725d45512.ba55ae97fe6d26f2fd2a296837570536->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf2/c332f2f6f5a44943a5e8c97725d45512-top, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/cf2/c845912c71644277a0aed468d0621d59.ba55ae97fe6d26f2fd2a296837570536->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf2/c845912c71644277a0aed468d0621d59-top, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/cf2/14e73a6c8cd44ed2ab40f783dad882ca.ba55ae97fe6d26f2fd2a296837570536->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf2/14e73a6c8cd44ed2ab40f783dad882ca-top, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/cf2/c4912c4098564070a195e3fd918ecec2.ba55ae97fe6d26f2fd2a296837570536->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf2/c4912c4098564070a195e3fd918ecec2-top, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/cf2/34f3b8d7d8774320929c1c1bbcc1f132.ba55ae97fe6d26f2fd2a296837570536->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf2/34f3b8d7d8774320929c1c1bbcc1f132-top] to archive 2023-05-10 04:55:22,263 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1683694518241.1b26b3dca87c03702b1dc11207d7b72a.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-05-10 04:55:22,264 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1683694518241.1b26b3dca87c03702b1dc11207d7b72a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/cf2/eb76ae13368e4a23855b5680af181f32.ba55ae97fe6d26f2fd2a296837570536 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/cf2/eb76ae13368e4a23855b5680af181f32.ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:22,265 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1683694518241.1b26b3dca87c03702b1dc11207d7b72a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/cf2/1747520752eb459e951b7491aa7c62a3.ba55ae97fe6d26f2fd2a296837570536 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/cf2/1747520752eb459e951b7491aa7c62a3.ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:22,266 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1683694518241.1b26b3dca87c03702b1dc11207d7b72a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/cf2/c332f2f6f5a44943a5e8c97725d45512.ba55ae97fe6d26f2fd2a296837570536 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/cf2/c332f2f6f5a44943a5e8c97725d45512.ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:22,267 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1683694518241.1b26b3dca87c03702b1dc11207d7b72a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/cf2/c845912c71644277a0aed468d0621d59.ba55ae97fe6d26f2fd2a296837570536 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/cf2/c845912c71644277a0aed468d0621d59.ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:22,267 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1683694518241.1b26b3dca87c03702b1dc11207d7b72a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/cf2/14e73a6c8cd44ed2ab40f783dad882ca.ba55ae97fe6d26f2fd2a296837570536 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/cf2/14e73a6c8cd44ed2ab40f783dad882ca.ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:22,268 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1683694518241.1b26b3dca87c03702b1dc11207d7b72a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/cf2/c4912c4098564070a195e3fd918ecec2.ba55ae97fe6d26f2fd2a296837570536 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/cf2/c4912c4098564070a195e3fd918ecec2.ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:22,269 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1683694518241.1b26b3dca87c03702b1dc11207d7b72a.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/cf2/34f3b8d7d8774320929c1c1bbcc1f132.ba55ae97fe6d26f2fd2a296837570536 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/cf2/34f3b8d7d8774320929c1c1bbcc1f132.ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:22,272 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-05-10 04:55:22,272 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-10 04:55:22,272 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testRecoveryAndDoubleExecution,41,1683694518241.1b26b3dca87c03702b1dc11207d7b72a. 2023-05-10 04:55:22,272 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1b26b3dca87c03702b1dc11207d7b72a: 2023-05-10 04:55:22,274 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 1b26b3dca87c03702b1dc11207d7b72a 2023-05-10 04:55:22,275 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=121 updating hbase:meta row=1b26b3dca87c03702b1dc11207d7b72a, regionState=CLOSED 2023-05-10 04:55:22,275 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,41,1683694518241.1b26b3dca87c03702b1dc11207d7b72a.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1683694522275"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683694522275"}]},"ts":"1683694522275"} 2023-05-10 04:55:22,277 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=122, resume processing ppid=121 2023-05-10 04:55:22,277 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=122, ppid=121, state=SUCCESS; CloseRegionProcedure 1b26b3dca87c03702b1dc11207d7b72a, server=jenkins-hbase4.apache.org,36665,1683694419452 in 201 msec 2023-05-10 04:55:22,278 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=121, resume processing ppid=119 2023-05-10 04:55:22,278 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=121, ppid=119, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=1b26b3dca87c03702b1dc11207d7b72a, UNASSIGN in 204 msec 2023-05-10 04:55:22,279 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683694522279"}]},"ts":"1683694522279"} 2023-05-10 04:55:22,279 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=DISABLED in hbase:meta 2023-05-10 04:55:22,281 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testRecoveryAndDoubleExecution to state=DISABLED 2023-05-10 04:55:22,282 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=119, state=SUCCESS; DisableTableProcedure table=testRecoveryAndDoubleExecution in 214 msec 2023-05-10 04:55:22,372 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(1227): Checking to see if procedure is done pid=119 2023-05-10 04:55:22,372 INFO [Listener at localhost/45007] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testRecoveryAndDoubleExecution, procId: 119 completed 2023-05-10 04:55:22,373 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete testRecoveryAndDoubleExecution 2023-05-10 04:55:22,374 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] procedure2.ProcedureExecutor(1029): Stored pid=124, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-05-10 04:55:22,375 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=124, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-05-10 04:55:22,376 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=124, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-05-10 04:55:22,376 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(1227): Checking to see if procedure is done pid=124 2023-05-10 04:55:22,380 DEBUG [HFileArchiver-13] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:22,381 DEBUG [HFileArchiver-14] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9 2023-05-10 04:55:22,381 DEBUG [HFileArchiver-15] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a 2023-05-10 04:55:22,382 DEBUG [HFileArchiver-13] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf1, FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf2, FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/recovered.edits] 2023-05-10 04:55:22,383 DEBUG [HFileArchiver-14] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/48042d1ae86f00fdead71e71ad8292a9, FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/cf1, FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/cf2, FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/recovered.edits] 2023-05-10 04:55:22,383 DEBUG [HFileArchiver-15] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/1b26b3dca87c03702b1dc11207d7b72a, FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/cf1, FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/cf2, FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/recovered.edits] 2023-05-10 04:55:22,386 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/48042d1ae86f00fdead71e71ad8292a9/.regioninfo to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/48042d1ae86f00fdead71e71ad8292a9/.regioninfo 2023-05-10 04:55:22,386 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf1/17ee8f61ba9d4f678887f8a52b767dbb to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf1/17ee8f61ba9d4f678887f8a52b767dbb 2023-05-10 04:55:22,386 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/1b26b3dca87c03702b1dc11207d7b72a/.regioninfo to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/1b26b3dca87c03702b1dc11207d7b72a/.regioninfo 2023-05-10 04:55:22,387 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf1/3488050453f04078bc1365497a2e3f88 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf1/3488050453f04078bc1365497a2e3f88 2023-05-10 04:55:22,389 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf1/4e150d9797e44e02a343bd11ff7b3dc3 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf1/4e150d9797e44e02a343bd11ff7b3dc3 2023-05-10 04:55:22,391 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/48042d1ae86f00fdead71e71ad8292a9/cf1/17ee8f61ba9d4f678887f8a52b767dbb.ba55ae97fe6d26f2fd2a296837570536 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/48042d1ae86f00fdead71e71ad8292a9/cf1/17ee8f61ba9d4f678887f8a52b767dbb.ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:22,391 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/1b26b3dca87c03702b1dc11207d7b72a/cf1/17ee8f61ba9d4f678887f8a52b767dbb.ba55ae97fe6d26f2fd2a296837570536 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/1b26b3dca87c03702b1dc11207d7b72a/cf1/17ee8f61ba9d4f678887f8a52b767dbb.ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:22,391 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf1/aa810537cf12426fa1bcdd14225b68ee to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf1/aa810537cf12426fa1bcdd14225b68ee 2023-05-10 04:55:22,392 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/48042d1ae86f00fdead71e71ad8292a9/cf1/3488050453f04078bc1365497a2e3f88.ba55ae97fe6d26f2fd2a296837570536 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/48042d1ae86f00fdead71e71ad8292a9/cf1/3488050453f04078bc1365497a2e3f88.ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:22,392 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/1b26b3dca87c03702b1dc11207d7b72a/cf1/3488050453f04078bc1365497a2e3f88.ba55ae97fe6d26f2fd2a296837570536 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/1b26b3dca87c03702b1dc11207d7b72a/cf1/3488050453f04078bc1365497a2e3f88.ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:22,392 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf1/b3e182af00c84ddb81eb7331183b802e to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf1/b3e182af00c84ddb81eb7331183b802e 2023-05-10 04:55:22,397 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/1b26b3dca87c03702b1dc11207d7b72a/cf1/4e150d9797e44e02a343bd11ff7b3dc3.ba55ae97fe6d26f2fd2a296837570536 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/1b26b3dca87c03702b1dc11207d7b72a/cf1/4e150d9797e44e02a343bd11ff7b3dc3.ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:22,398 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/48042d1ae86f00fdead71e71ad8292a9/cf1/4e150d9797e44e02a343bd11ff7b3dc3.ba55ae97fe6d26f2fd2a296837570536 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/48042d1ae86f00fdead71e71ad8292a9/cf1/4e150d9797e44e02a343bd11ff7b3dc3.ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:22,398 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf1/bd5a17faa7564deb857bb9ec28719ee6 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf1/bd5a17faa7564deb857bb9ec28719ee6 2023-05-10 04:55:22,399 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/1b26b3dca87c03702b1dc11207d7b72a/cf1/aa810537cf12426fa1bcdd14225b68ee.ba55ae97fe6d26f2fd2a296837570536 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/1b26b3dca87c03702b1dc11207d7b72a/cf1/aa810537cf12426fa1bcdd14225b68ee.ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:22,399 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/48042d1ae86f00fdead71e71ad8292a9/cf1/aa810537cf12426fa1bcdd14225b68ee.ba55ae97fe6d26f2fd2a296837570536 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/48042d1ae86f00fdead71e71ad8292a9/cf1/aa810537cf12426fa1bcdd14225b68ee.ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:22,399 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf1/c2d599f271fb497b8b272a3bfd391f6e to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf1/c2d599f271fb497b8b272a3bfd391f6e 2023-05-10 04:55:22,400 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/1b26b3dca87c03702b1dc11207d7b72a/cf1/b3e182af00c84ddb81eb7331183b802e.ba55ae97fe6d26f2fd2a296837570536 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/1b26b3dca87c03702b1dc11207d7b72a/cf1/b3e182af00c84ddb81eb7331183b802e.ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:22,400 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/48042d1ae86f00fdead71e71ad8292a9/cf1/b3e182af00c84ddb81eb7331183b802e.ba55ae97fe6d26f2fd2a296837570536 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/48042d1ae86f00fdead71e71ad8292a9/cf1/b3e182af00c84ddb81eb7331183b802e.ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:22,401 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/1b26b3dca87c03702b1dc11207d7b72a/cf1/bd5a17faa7564deb857bb9ec28719ee6.ba55ae97fe6d26f2fd2a296837570536 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/1b26b3dca87c03702b1dc11207d7b72a/cf1/bd5a17faa7564deb857bb9ec28719ee6.ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:22,402 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/48042d1ae86f00fdead71e71ad8292a9/cf1/bd5a17faa7564deb857bb9ec28719ee6.ba55ae97fe6d26f2fd2a296837570536 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/48042d1ae86f00fdead71e71ad8292a9/cf1/bd5a17faa7564deb857bb9ec28719ee6.ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:22,402 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf2/14e73a6c8cd44ed2ab40f783dad882ca to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf2/14e73a6c8cd44ed2ab40f783dad882ca 2023-05-10 04:55:22,403 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/1b26b3dca87c03702b1dc11207d7b72a/cf1/c2d599f271fb497b8b272a3bfd391f6e.ba55ae97fe6d26f2fd2a296837570536 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/1b26b3dca87c03702b1dc11207d7b72a/cf1/c2d599f271fb497b8b272a3bfd391f6e.ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:22,403 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/48042d1ae86f00fdead71e71ad8292a9/cf1/c2d599f271fb497b8b272a3bfd391f6e.ba55ae97fe6d26f2fd2a296837570536 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/48042d1ae86f00fdead71e71ad8292a9/cf1/c2d599f271fb497b8b272a3bfd391f6e.ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:22,404 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf2/1747520752eb459e951b7491aa7c62a3 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf2/1747520752eb459e951b7491aa7c62a3 2023-05-10 04:55:22,405 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf2/34f3b8d7d8774320929c1c1bbcc1f132 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf2/34f3b8d7d8774320929c1c1bbcc1f132 2023-05-10 04:55:22,406 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/1b26b3dca87c03702b1dc11207d7b72a/cf2/14e73a6c8cd44ed2ab40f783dad882ca.ba55ae97fe6d26f2fd2a296837570536 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/1b26b3dca87c03702b1dc11207d7b72a/cf2/14e73a6c8cd44ed2ab40f783dad882ca.ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:22,406 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/48042d1ae86f00fdead71e71ad8292a9/cf2/14e73a6c8cd44ed2ab40f783dad882ca.ba55ae97fe6d26f2fd2a296837570536 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/48042d1ae86f00fdead71e71ad8292a9/cf2/14e73a6c8cd44ed2ab40f783dad882ca.ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:22,407 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf2/c332f2f6f5a44943a5e8c97725d45512 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf2/c332f2f6f5a44943a5e8c97725d45512 2023-05-10 04:55:22,407 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/1b26b3dca87c03702b1dc11207d7b72a/cf2/1747520752eb459e951b7491aa7c62a3.ba55ae97fe6d26f2fd2a296837570536 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/1b26b3dca87c03702b1dc11207d7b72a/cf2/1747520752eb459e951b7491aa7c62a3.ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:22,408 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/48042d1ae86f00fdead71e71ad8292a9/cf2/1747520752eb459e951b7491aa7c62a3.ba55ae97fe6d26f2fd2a296837570536 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/48042d1ae86f00fdead71e71ad8292a9/cf2/1747520752eb459e951b7491aa7c62a3.ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:22,408 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf2/c4912c4098564070a195e3fd918ecec2 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf2/c4912c4098564070a195e3fd918ecec2 2023-05-10 04:55:22,408 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/1b26b3dca87c03702b1dc11207d7b72a/cf2/34f3b8d7d8774320929c1c1bbcc1f132.ba55ae97fe6d26f2fd2a296837570536 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/1b26b3dca87c03702b1dc11207d7b72a/cf2/34f3b8d7d8774320929c1c1bbcc1f132.ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:22,409 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/48042d1ae86f00fdead71e71ad8292a9/cf2/34f3b8d7d8774320929c1c1bbcc1f132.ba55ae97fe6d26f2fd2a296837570536 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/48042d1ae86f00fdead71e71ad8292a9/cf2/34f3b8d7d8774320929c1c1bbcc1f132.ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:22,409 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf2/c845912c71644277a0aed468d0621d59 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf2/c845912c71644277a0aed468d0621d59 2023-05-10 04:55:22,411 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/1b26b3dca87c03702b1dc11207d7b72a/cf2/c332f2f6f5a44943a5e8c97725d45512.ba55ae97fe6d26f2fd2a296837570536 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/1b26b3dca87c03702b1dc11207d7b72a/cf2/c332f2f6f5a44943a5e8c97725d45512.ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:22,411 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/48042d1ae86f00fdead71e71ad8292a9/cf2/c332f2f6f5a44943a5e8c97725d45512.ba55ae97fe6d26f2fd2a296837570536 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/48042d1ae86f00fdead71e71ad8292a9/cf2/c332f2f6f5a44943a5e8c97725d45512.ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:22,412 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf2/eb76ae13368e4a23855b5680af181f32 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/cf2/eb76ae13368e4a23855b5680af181f32 2023-05-10 04:55:22,412 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/1b26b3dca87c03702b1dc11207d7b72a/cf2/c4912c4098564070a195e3fd918ecec2.ba55ae97fe6d26f2fd2a296837570536 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/1b26b3dca87c03702b1dc11207d7b72a/cf2/c4912c4098564070a195e3fd918ecec2.ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:22,412 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/48042d1ae86f00fdead71e71ad8292a9/cf2/c4912c4098564070a195e3fd918ecec2.ba55ae97fe6d26f2fd2a296837570536 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/48042d1ae86f00fdead71e71ad8292a9/cf2/c4912c4098564070a195e3fd918ecec2.ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:22,413 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/1b26b3dca87c03702b1dc11207d7b72a/cf2/c845912c71644277a0aed468d0621d59.ba55ae97fe6d26f2fd2a296837570536 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/1b26b3dca87c03702b1dc11207d7b72a/cf2/c845912c71644277a0aed468d0621d59.ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:22,414 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/48042d1ae86f00fdead71e71ad8292a9/cf2/c845912c71644277a0aed468d0621d59.ba55ae97fe6d26f2fd2a296837570536 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/48042d1ae86f00fdead71e71ad8292a9/cf2/c845912c71644277a0aed468d0621d59.ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:22,414 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/recovered.edits/85.seqid to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536/recovered.edits/85.seqid 2023-05-10 04:55:22,415 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/1b26b3dca87c03702b1dc11207d7b72a/cf2/eb76ae13368e4a23855b5680af181f32.ba55ae97fe6d26f2fd2a296837570536 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/1b26b3dca87c03702b1dc11207d7b72a/cf2/eb76ae13368e4a23855b5680af181f32.ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:22,415 DEBUG [HFileArchiver-13] backup.HFileArchiver(596): Deleted hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testRecoveryAndDoubleExecution/ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:22,415 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/48042d1ae86f00fdead71e71ad8292a9/cf2/eb76ae13368e4a23855b5680af181f32.ba55ae97fe6d26f2fd2a296837570536 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/48042d1ae86f00fdead71e71ad8292a9/cf2/eb76ae13368e4a23855b5680af181f32.ba55ae97fe6d26f2fd2a296837570536 2023-05-10 04:55:22,418 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/cf1/e7f43908ebe941f5b96351fbdfd3db7d to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/cf1/e7f43908ebe941f5b96351fbdfd3db7d 2023-05-10 04:55:22,419 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/cf1/e8d699bc79c24348a272b5b24f3d26d2 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/cf1/e8d699bc79c24348a272b5b24f3d26d2 2023-05-10 04:55:22,421 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/cf2/53c2352e58614b309a0a33f58376937f to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/cf2/53c2352e58614b309a0a33f58376937f 2023-05-10 04:55:22,422 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/cf2/a804054f1e484a71ac9718acf12a6a3c to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/cf2/a804054f1e484a71ac9718acf12a6a3c 2023-05-10 04:55:22,425 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/recovered.edits/90.seqid to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a/recovered.edits/90.seqid 2023-05-10 04:55:22,426 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/recovered.edits/90.seqid to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9/recovered.edits/90.seqid 2023-05-10 04:55:22,426 DEBUG [HFileArchiver-15] backup.HFileArchiver(596): Deleted hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testRecoveryAndDoubleExecution/1b26b3dca87c03702b1dc11207d7b72a 2023-05-10 04:55:22,427 DEBUG [HFileArchiver-14] backup.HFileArchiver(596): Deleted hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testRecoveryAndDoubleExecution/48042d1ae86f00fdead71e71ad8292a9 2023-05-10 04:55:22,427 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRecoveryAndDoubleExecution regions 2023-05-10 04:55:22,429 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=124, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-05-10 04:55:22,431 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testRecoveryAndDoubleExecution from hbase:meta 2023-05-10 04:55:22,432 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testRecoveryAndDoubleExecution' descriptor. 2023-05-10 04:55:22,433 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=124, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-05-10 04:55:22,433 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testRecoveryAndDoubleExecution' from region states. 2023-05-10 04:55:22,433 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution,,1683694516443.ba55ae97fe6d26f2fd2a296837570536.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1683694522433"}]},"ts":"9223372036854775807"} 2023-05-10 04:55:22,433 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution,,1683694518241.48042d1ae86f00fdead71e71ad8292a9.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1683694522433"}]},"ts":"9223372036854775807"} 2023-05-10 04:55:22,433 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution,41,1683694518241.1b26b3dca87c03702b1dc11207d7b72a.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1683694522433"}]},"ts":"9223372036854775807"} 2023-05-10 04:55:22,435 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-05-10 04:55:22,435 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => ba55ae97fe6d26f2fd2a296837570536, NAME => 'testRecoveryAndDoubleExecution,,1683694516443.ba55ae97fe6d26f2fd2a296837570536.', STARTKEY => '', ENDKEY => '', OFFLINE => true, SPLIT => true}, {ENCODED => 48042d1ae86f00fdead71e71ad8292a9, NAME => 'testRecoveryAndDoubleExecution,,1683694518241.48042d1ae86f00fdead71e71ad8292a9.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => 1b26b3dca87c03702b1dc11207d7b72a, NAME => 'testRecoveryAndDoubleExecution,41,1683694518241.1b26b3dca87c03702b1dc11207d7b72a.', STARTKEY => '41', ENDKEY => ''}] 2023-05-10 04:55:22,435 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testRecoveryAndDoubleExecution' as deleted. 2023-05-10 04:55:22,435 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1683694522435"}]},"ts":"9223372036854775807"} 2023-05-10 04:55:22,436 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testRecoveryAndDoubleExecution state from META 2023-05-10 04:55:22,438 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=124, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-05-10 04:55:22,439 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=124, state=SUCCESS; DeleteTableProcedure table=testRecoveryAndDoubleExecution in 65 msec 2023-05-10 04:55:22,477 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(1227): Checking to see if procedure is done pid=124 2023-05-10 04:55:22,477 INFO [Listener at localhost/45007] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testRecoveryAndDoubleExecution, procId: 124 completed 2023-05-10 04:55:22,494 INFO [Listener at localhost/45007] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testRecoveryAndDoubleExecution Thread=377 (was 368) - Thread LEAK? -, OpenFileDescriptor=835 (was 744) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=171 (was 177), ProcessCount=174 (was 172) - ProcessCount LEAK? -, AvailableMemoryMB=9537 (was 9552) 2023-05-10 04:55:22,510 INFO [Listener at localhost/45007] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionEmptyDaughter Thread=377, OpenFileDescriptor=835, MaxFileDescriptor=60000, SystemLoadAverage=171, ProcessCount=174, AvailableMemoryMB=9537 2023-05-10 04:55:22,510 WARN [Listener at localhost/45007] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-05-10 04:55:22,514 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-05-10 04:55:22,514 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-05-10 04:55:22,516 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(1029): Stored pid=125, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionEmptyDaughter 2023-05-10 04:55:22,517 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_PRE_OPERATION 2023-05-10 04:55:22,518 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-05-10 04:55:22,519 DEBUG [HFileArchiver-16] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292 2023-05-10 04:55:22,519 DEBUG [HFileArchiver-16] backup.HFileArchiver(153): Directory hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292 empty. 2023-05-10 04:55:22,520 DEBUG [HFileArchiver-16] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292 2023-05-10 04:55:22,520 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionEmptyDaughter regions 2023-05-10 04:55:22,528 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionEmptyDaughter/.tabledesc/.tableinfo.0000000001 2023-05-10 04:55:22,529 INFO [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(7675): creating {ENCODED => 4fb74b802ab843984094c342c1179292, NAME => 'testSplitTableRegionEmptyDaughter,,1683694522515.4fb74b802ab843984094c342c1179292.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionEmptyDaughter', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp 2023-05-10 04:55:22,535 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(866): Instantiated testSplitTableRegionEmptyDaughter,,1683694522515.4fb74b802ab843984094c342c1179292.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-10 04:55:22,535 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1604): Closing 4fb74b802ab843984094c342c1179292, disabling compactions & flushes 2023-05-10 04:55:22,536 INFO [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1626): Closing region testSplitTableRegionEmptyDaughter,,1683694522515.4fb74b802ab843984094c342c1179292. 2023-05-10 04:55:22,536 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,,1683694522515.4fb74b802ab843984094c342c1179292. 2023-05-10 04:55:22,536 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionEmptyDaughter,,1683694522515.4fb74b802ab843984094c342c1179292. after waiting 0 ms 2023-05-10 04:55:22,536 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionEmptyDaughter,,1683694522515.4fb74b802ab843984094c342c1179292. 2023-05-10 04:55:22,536 INFO [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1838): Closed testSplitTableRegionEmptyDaughter,,1683694522515.4fb74b802ab843984094c342c1179292. 2023-05-10 04:55:22,536 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1558): Region close journal for 4fb74b802ab843984094c342c1179292: 2023-05-10 04:55:22,537 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_ADD_TO_META 2023-05-10 04:55:22,538 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,,1683694522515.4fb74b802ab843984094c342c1179292.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1683694522538"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683694522538"}]},"ts":"1683694522538"} 2023-05-10 04:55:22,539 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-05-10 04:55:22,540 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-05-10 04:55:22,540 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683694522540"}]},"ts":"1683694522540"} 2023-05-10 04:55:22,541 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=ENABLING in hbase:meta 2023-05-10 04:55:22,543 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-05-10 04:55:22,544 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-05-10 04:55:22,544 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-05-10 04:55:22,544 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-05-10 04:55:22,544 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-05-10 04:55:22,544 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=126, ppid=125, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=4fb74b802ab843984094c342c1179292, ASSIGN}] 2023-05-10 04:55:22,545 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=126, ppid=125, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=4fb74b802ab843984094c342c1179292, ASSIGN 2023-05-10 04:55:22,546 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=126, ppid=125, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=4fb74b802ab843984094c342c1179292, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,36665,1683694419452; forceNewPlan=false, retain=false 2023-05-10 04:55:22,696 INFO [jenkins-hbase4:37903] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-05-10 04:55:22,697 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=126 updating hbase:meta row=4fb74b802ab843984094c342c1179292, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:22,697 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1683694522515.4fb74b802ab843984094c342c1179292.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1683694522697"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683694522697"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683694522697"}]},"ts":"1683694522697"} 2023-05-10 04:55:22,698 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=127, ppid=126, state=RUNNABLE; OpenRegionProcedure 4fb74b802ab843984094c342c1179292, server=jenkins-hbase4.apache.org,36665,1683694419452}] 2023-05-10 04:55:22,852 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionEmptyDaughter,,1683694522515.4fb74b802ab843984094c342c1179292. 2023-05-10 04:55:22,852 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 4fb74b802ab843984094c342c1179292, NAME => 'testSplitTableRegionEmptyDaughter,,1683694522515.4fb74b802ab843984094c342c1179292.', STARTKEY => '', ENDKEY => ''} 2023-05-10 04:55:22,853 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-05-10 04:55:22,853 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionEmptyDaughter 4fb74b802ab843984094c342c1179292 2023-05-10 04:55:22,853 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionEmptyDaughter,,1683694522515.4fb74b802ab843984094c342c1179292.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-10 04:55:22,853 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 4fb74b802ab843984094c342c1179292 2023-05-10 04:55:22,853 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 4fb74b802ab843984094c342c1179292 2023-05-10 04:55:22,854 INFO [StoreOpener-4fb74b802ab843984094c342c1179292-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 4fb74b802ab843984094c342c1179292 2023-05-10 04:55:22,855 DEBUG [StoreOpener-4fb74b802ab843984094c342c1179292-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/cf1 2023-05-10 04:55:22,855 DEBUG [StoreOpener-4fb74b802ab843984094c342c1179292-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/cf1 2023-05-10 04:55:22,856 INFO [StoreOpener-4fb74b802ab843984094c342c1179292-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 4fb74b802ab843984094c342c1179292 columnFamilyName cf1 2023-05-10 04:55:22,856 INFO [StoreOpener-4fb74b802ab843984094c342c1179292-1] regionserver.HStore(310): Store=4fb74b802ab843984094c342c1179292/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-10 04:55:22,856 INFO [StoreOpener-4fb74b802ab843984094c342c1179292-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 4fb74b802ab843984094c342c1179292 2023-05-10 04:55:22,857 DEBUG [StoreOpener-4fb74b802ab843984094c342c1179292-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/cf2 2023-05-10 04:55:22,857 DEBUG [StoreOpener-4fb74b802ab843984094c342c1179292-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/cf2 2023-05-10 04:55:22,857 INFO [StoreOpener-4fb74b802ab843984094c342c1179292-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 4fb74b802ab843984094c342c1179292 columnFamilyName cf2 2023-05-10 04:55:22,858 INFO [StoreOpener-4fb74b802ab843984094c342c1179292-1] regionserver.HStore(310): Store=4fb74b802ab843984094c342c1179292/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-10 04:55:22,858 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292 2023-05-10 04:55:22,859 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292 2023-05-10 04:55:22,860 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionEmptyDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-05-10 04:55:22,861 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 4fb74b802ab843984094c342c1179292 2023-05-10 04:55:22,862 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-05-10 04:55:22,863 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 4fb74b802ab843984094c342c1179292; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72191823, jitterRate=0.07574199140071869}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-05-10 04:55:22,863 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 4fb74b802ab843984094c342c1179292: 2023-05-10 04:55:22,863 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2334): Post open deploy tasks for testSplitTableRegionEmptyDaughter,,1683694522515.4fb74b802ab843984094c342c1179292., pid=127, masterSystemTime=1683694522850 2023-05-10 04:55:22,865 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2361): Finished post open deploy task for testSplitTableRegionEmptyDaughter,,1683694522515.4fb74b802ab843984094c342c1179292. 2023-05-10 04:55:22,865 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionEmptyDaughter,,1683694522515.4fb74b802ab843984094c342c1179292. 2023-05-10 04:55:22,865 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=126 updating hbase:meta row=4fb74b802ab843984094c342c1179292, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:22,865 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionEmptyDaughter,,1683694522515.4fb74b802ab843984094c342c1179292.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1683694522865"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1683694522865"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1683694522865"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683694522865"}]},"ts":"1683694522865"} 2023-05-10 04:55:22,868 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=127, resume processing ppid=126 2023-05-10 04:55:22,868 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=127, ppid=126, state=SUCCESS; OpenRegionProcedure 4fb74b802ab843984094c342c1179292, server=jenkins-hbase4.apache.org,36665,1683694419452 in 169 msec 2023-05-10 04:55:22,869 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=126, resume processing ppid=125 2023-05-10 04:55:22,869 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=126, ppid=125, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=4fb74b802ab843984094c342c1179292, ASSIGN in 324 msec 2023-05-10 04:55:22,870 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-05-10 04:55:22,870 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683694522870"}]},"ts":"1683694522870"} 2023-05-10 04:55:22,871 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=ENABLED in hbase:meta 2023-05-10 04:55:22,873 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_POST_OPERATION 2023-05-10 04:55:22,874 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=125, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionEmptyDaughter in 358 msec 2023-05-10 04:55:22,975 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegionEmptyDaughter' 2023-05-10 04:55:23,023 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-10 04:55:23,024 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,024 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-05-10 04:55:23,024 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:55:23,024 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-05-10 04:55:23,024 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-05-10 04:55:23,024 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,025 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,026 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:23,027 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:23,027 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:23,027 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:23,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:23,027 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:23,027 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:23,027 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:23,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:23,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:23,027 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:23,027 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-10 04:55:23,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,027 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,027 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,027 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,027 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:55:23,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:55:23,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:55:23,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,028 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,028 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,028 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,028 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,028 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,028 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-05-10 04:55:23,028 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:55:23,028 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,028 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,028 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-05-10 04:55:23,028 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-05-10 04:55:23,028 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-05-10 04:55:23,029 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:55:23,029 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-05-10 04:55:23,029 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-05-10 04:55:23,029 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-05-10 04:55:23,028 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:55:23,029 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37823,1683694419534' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-05-10 04:55:23,029 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37995,1683694419499' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-05-10 04:55:23,029 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-05-10 04:55:23,029 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-10 04:55:23,029 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-10 04:55:23,029 DEBUG [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1683694522515.4fb74b802ab843984094c342c1179292. 2023-05-10 04:55:23,029 DEBUG [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1683694522515.4fb74b802ab843984094c342c1179292. started... 2023-05-10 04:55:23,030 INFO [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 4fb74b802ab843984094c342c1179292 2/2 column families, dataSize=120 B heapSize=928 B 2023-05-10 04:55:23,031 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,031 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,031 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,031 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:55:23,031 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,031 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:55:23,038 INFO [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/.tmp/cf1/fef4b1b4b5fe46a0bba604884b9694e3 2023-05-10 04:55:23,049 INFO [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/.tmp/cf2/8bfce04d38894ab1b6d10a9aef11f8a8 2023-05-10 04:55:23,052 DEBUG [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/.tmp/cf1/fef4b1b4b5fe46a0bba604884b9694e3 as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/cf1/fef4b1b4b5fe46a0bba604884b9694e3 2023-05-10 04:55:23,055 INFO [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/cf1/fef4b1b4b5fe46a0bba604884b9694e3, entries=2, sequenceid=6, filesize=4.8 K 2023-05-10 04:55:23,056 DEBUG [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/.tmp/cf2/8bfce04d38894ab1b6d10a9aef11f8a8 as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/cf2/8bfce04d38894ab1b6d10a9aef11f8a8 2023-05-10 04:55:23,059 INFO [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/cf2/8bfce04d38894ab1b6d10a9aef11f8a8, entries=2, sequenceid=6, filesize=4.8 K 2023-05-10 04:55:23,059 INFO [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 4fb74b802ab843984094c342c1179292 in 29ms, sequenceid=6, compaction requested=false 2023-05-10 04:55:23,060 DEBUG [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 4fb74b802ab843984094c342c1179292: 2023-05-10 04:55:23,060 DEBUG [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1683694522515.4fb74b802ab843984094c342c1179292. 2023-05-10 04:55:23,060 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-10 04:55:23,061 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-10 04:55:23,061 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:23,061 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-05-10 04:55:23,061 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36665,1683694419452' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-05-10 04:55:23,062 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,062 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:23,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:23,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:55:23,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:55:23,062 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,062 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:55:23,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:55:23,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:55:23,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:23,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:23,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:23,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:55:23,064 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36665,1683694419452' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-05-10 04:55:23,064 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1651b8d3[Count = 0] remaining members to acquire global barrier 2023-05-10 04:55:23,064 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-05-10 04:55:23,064 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,065 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,065 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,065 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,065 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,065 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-05-10 04:55:23,065 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,065 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-05-10 04:55:23,065 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,37995,1683694419499' in zk 2023-05-10 04:55:23,065 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,065 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:23,065 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-10 04:55:23,065 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-05-10 04:55:23,066 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-05-10 04:55:23,066 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,36665,1683694419452' in zk 2023-05-10 04:55:23,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,066 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-05-10 04:55:23,066 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-05-10 04:55:23,066 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,37823,1683694419534' in zk 2023-05-10 04:55:23,068 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-05-10 04:55:23,068 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:55:23,068 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-05-10 04:55:23,069 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-05-10 04:55:23,069 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:23,069 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:55:23,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:23,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:55:23,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:55:23,069 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-05-10 04:55:23,069 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:55:23,069 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-05-10 04:55:23,069 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-05-10 04:55:23,069 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:55:23,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:55:23,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:23,070 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:23,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:23,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:55:23,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:23,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:23,071 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:23,072 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,36665,1683694419452': 2023-05-10 04:55:23,072 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36665,1683694419452' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-05-10 04:55:23,072 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-05-10 04:55:23,072 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-10 04:55:23,072 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-10 04:55:23,072 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,072 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-10 04:55:23,073 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,073 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,073 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,073 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:55:23,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:55:23,073 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,073 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:23,074 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,074 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:23,074 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:55:23,074 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:23,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,074 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:23,074 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:23,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:23,074 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:23,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:23,074 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:23,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,074 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:23,074 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:23,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:55:23,075 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:23,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,075 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:23,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:23,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:23,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:55:23,076 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:23,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,076 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:23,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:23,077 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:23,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:23,077 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:23,085 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:23,086 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:23,086 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,086 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:23,086 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:23,086 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:23,086 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-05-10 04:55:23,086 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-10 04:55:23,086 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:23,086 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:23,086 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:23,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:23,086 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:55:23,086 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-05-10 04:55:23,086 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:23,086 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-10 04:55:23,086 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:23,086 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:23,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:23,086 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:23,086 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:23,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:23,086 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:23,087 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:23,087 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,087 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:23,087 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:23,087 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,087 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,087 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:23,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:23,087 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:23,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:23,186 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-10 04:55:23,187 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-10 04:55:23,197 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-10 04:55:23,199 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-05-10 04:55:23,199 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,199 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-05-10 04:55:23,199 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:55:23,199 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-05-10 04:55:23,199 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-05-10 04:55:23,199 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,199 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,200 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:23,200 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:23,200 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:23,200 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:23,200 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:23,200 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:23,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:23,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:23,201 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:23,201 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-10 04:55:23,201 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:23,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:23,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,201 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,201 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,201 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,201 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:55:23,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:55:23,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:55:23,202 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,202 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,202 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,202 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,202 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-05-10 04:55:23,202 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-05-10 04:55:23,202 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-05-10 04:55:23,202 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,202 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,202 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-05-10 04:55:23,202 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-05-10 04:55:23,202 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:55:23,202 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,202 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-05-10 04:55:23,203 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-05-10 04:55:23,203 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:55:23,202 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:55:23,203 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-05-10 04:55:23,203 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-05-10 04:55:23,203 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-05-10 04:55:23,203 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-05-10 04:55:23,203 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37995,1683694419499' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-05-10 04:55:23,203 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37823,1683694419534' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-05-10 04:55:23,203 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-10 04:55:23,203 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-10 04:55:23,203 DEBUG [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1683694522515.4fb74b802ab843984094c342c1179292. 2023-05-10 04:55:23,203 DEBUG [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1683694522515.4fb74b802ab843984094c342c1179292. started... 2023-05-10 04:55:23,203 INFO [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 4fb74b802ab843984094c342c1179292 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-10 04:55:23,204 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,204 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,204 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,204 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,204 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:55:23,204 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:55:23,211 INFO [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/.tmp/cf1/ecc198964bd246519840a47b05401987 2023-05-10 04:55:23,221 INFO [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/.tmp/cf2/37ad6bf54a7f4109a4fa74e3d9607266 2023-05-10 04:55:23,224 DEBUG [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/.tmp/cf1/ecc198964bd246519840a47b05401987 as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/cf1/ecc198964bd246519840a47b05401987 2023-05-10 04:55:23,227 INFO [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/cf1/ecc198964bd246519840a47b05401987, entries=10, sequenceid=19, filesize=5.0 K 2023-05-10 04:55:23,228 DEBUG [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/.tmp/cf2/37ad6bf54a7f4109a4fa74e3d9607266 as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/cf2/37ad6bf54a7f4109a4fa74e3d9607266 2023-05-10 04:55:23,231 INFO [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/cf2/37ad6bf54a7f4109a4fa74e3d9607266, entries=10, sequenceid=19, filesize=5.0 K 2023-05-10 04:55:23,232 INFO [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 4fb74b802ab843984094c342c1179292 in 29ms, sequenceid=19, compaction requested=false 2023-05-10 04:55:23,232 DEBUG [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 4fb74b802ab843984094c342c1179292: 2023-05-10 04:55:23,232 DEBUG [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1683694522515.4fb74b802ab843984094c342c1179292. 2023-05-10 04:55:23,232 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-10 04:55:23,232 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-10 04:55:23,232 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:23,232 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-05-10 04:55:23,232 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36665,1683694419452' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-05-10 04:55:23,234 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:23,234 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:23,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:55:23,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:55:23,234 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:55:23,234 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:55:23,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:55:23,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:23,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:23,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:23,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:55:23,236 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36665,1683694419452' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-05-10 04:55:23,236 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4a6ab8e4[Count = 0] remaining members to acquire global barrier 2023-05-10 04:55:23,236 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-05-10 04:55:23,236 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,238 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,238 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,238 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,238 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,238 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:23,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,238 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,238 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-05-10 04:55:23,239 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,238 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-10 04:55:23,238 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,239 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,239 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-05-10 04:55:23,239 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-05-10 04:55:23,239 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-05-10 04:55:23,239 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,36665,1683694419452' in zk 2023-05-10 04:55:23,239 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-05-10 04:55:23,239 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-05-10 04:55:23,239 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,37823,1683694419534' in zk 2023-05-10 04:55:23,239 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,37995,1683694419499' in zk 2023-05-10 04:55:23,240 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-05-10 04:55:23,240 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-05-10 04:55:23,240 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-05-10 04:55:23,240 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:55:23,240 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:55:23,240 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:23,240 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-05-10 04:55:23,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:23,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:55:23,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:55:23,240 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-05-10 04:55:23,240 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:55:23,240 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-05-10 04:55:23,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:55:23,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:55:23,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:23,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:23,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:23,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:55:23,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:23,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:23,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:23,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,36665,1683694419452': 2023-05-10 04:55:23,243 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36665,1683694419452' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-05-10 04:55:23,243 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-05-10 04:55:23,243 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-10 04:55:23,243 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-10 04:55:23,243 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,243 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-10 04:55:23,245 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,245 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,245 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,245 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:23,245 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,245 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:23,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:55:23,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:55:23,245 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,245 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,245 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:23,245 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,245 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:23,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:23,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:55:23,245 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:23,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,246 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:23,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,246 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:23,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:23,246 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:23,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:23,246 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:23,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:55:23,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:23,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:23,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:23,248 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:23,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:55:23,248 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:23,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,248 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:23,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:23,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:23,249 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:23,251 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:23,251 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:23,251 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:23,251 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:23,251 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:23,251 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:23,251 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:23,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:23,251 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:55:23,251 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-05-10 04:55:23,251 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-10 04:55:23,251 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:23,251 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:23,251 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,251 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:23,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:23,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:23,251 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-05-10 04:55:23,251 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:23,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:23,251 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:23,251 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:23,251 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-10 04:55:23,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:23,251 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:23,251 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:23,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:23,252 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,252 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:23,252 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,252 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:23,252 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:23,252 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,252 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,352 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-10 04:55:23,353 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-10 04:55:23,363 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-10 04:55:23,364 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-05-10 04:55:23,364 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,364 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-05-10 04:55:23,365 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:55:23,365 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-05-10 04:55:23,365 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-05-10 04:55:23,365 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,365 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,368 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:23,368 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:23,368 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:23,368 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:23,368 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:23,368 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:23,368 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:23,369 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-10 04:55:23,368 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:23,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:23,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:23,368 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:23,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,369 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,369 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,369 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,369 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:55:23,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:55:23,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,370 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:55:23,370 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,370 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,370 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,370 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-05-10 04:55:23,370 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,370 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-05-10 04:55:23,370 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:55:23,370 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-05-10 04:55:23,370 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-05-10 04:55:23,370 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-05-10 04:55:23,370 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37995,1683694419499' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-05-10 04:55:23,370 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,371 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-05-10 04:55:23,371 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:55:23,371 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-05-10 04:55:23,372 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,372 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,372 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:55:23,374 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-05-10 04:55:23,375 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,375 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-10 04:55:23,375 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-05-10 04:55:23,375 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:55:23,375 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-10 04:55:23,375 DEBUG [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1683694522515.4fb74b802ab843984094c342c1179292. 2023-05-10 04:55:23,375 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-05-10 04:55:23,375 DEBUG [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1683694522515.4fb74b802ab843984094c342c1179292. started... 2023-05-10 04:55:23,375 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-05-10 04:55:23,375 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37823,1683694419534' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-05-10 04:55:23,375 INFO [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 4fb74b802ab843984094c342c1179292 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-10 04:55:23,376 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,376 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,377 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:55:23,388 INFO [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/.tmp/cf1/f0e775b023894593afde5f0d4ff3f2e5 2023-05-10 04:55:23,398 INFO [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/.tmp/cf2/1ebabb90982c4266a853080b40b2d69c 2023-05-10 04:55:23,402 DEBUG [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/.tmp/cf1/f0e775b023894593afde5f0d4ff3f2e5 as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/cf1/f0e775b023894593afde5f0d4ff3f2e5 2023-05-10 04:55:23,405 INFO [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/cf1/f0e775b023894593afde5f0d4ff3f2e5, entries=10, sequenceid=32, filesize=5.0 K 2023-05-10 04:55:23,405 DEBUG [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/.tmp/cf2/1ebabb90982c4266a853080b40b2d69c as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/cf2/1ebabb90982c4266a853080b40b2d69c 2023-05-10 04:55:23,409 INFO [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/cf2/1ebabb90982c4266a853080b40b2d69c, entries=10, sequenceid=32, filesize=5.0 K 2023-05-10 04:55:23,409 INFO [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 4fb74b802ab843984094c342c1179292 in 34ms, sequenceid=32, compaction requested=true 2023-05-10 04:55:23,409 DEBUG [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 4fb74b802ab843984094c342c1179292: 2023-05-10 04:55:23,410 DEBUG [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1683694522515.4fb74b802ab843984094c342c1179292. 2023-05-10 04:55:23,410 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-10 04:55:23,410 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-10 04:55:23,410 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:23,410 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-05-10 04:55:23,410 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36665,1683694419452' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-05-10 04:55:23,411 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:23,411 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:23,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:55:23,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:55:23,411 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,411 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:55:23,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:55:23,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:55:23,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:23,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:23,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:23,413 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:55:23,413 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36665,1683694419452' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-05-10 04:55:23,413 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7b4bfbea[Count = 0] remaining members to acquire global barrier 2023-05-10 04:55:23,413 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-05-10 04:55:23,413 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,416 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,416 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,416 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,416 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,416 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,416 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-05-10 04:55:23,416 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,416 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:23,416 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-10 04:55:23,416 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,416 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-05-10 04:55:23,416 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-05-10 04:55:23,416 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-05-10 04:55:23,416 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,37995,1683694419499' in zk 2023-05-10 04:55:23,416 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-05-10 04:55:23,416 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-05-10 04:55:23,416 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,36665,1683694419452' in zk 2023-05-10 04:55:23,416 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,37823,1683694419534' in zk 2023-05-10 04:55:23,420 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-05-10 04:55:23,420 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-05-10 04:55:23,420 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:23,420 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:55:23,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:23,420 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:55:23,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:55:23,420 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-05-10 04:55:23,421 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:55:23,420 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:55:23,421 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-05-10 04:55:23,420 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-05-10 04:55:23,421 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-05-10 04:55:23,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:55:23,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:55:23,421 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:23,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:23,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:23,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:55:23,422 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:23,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:23,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:23,423 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,36665,1683694419452': 2023-05-10 04:55:23,423 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36665,1683694419452' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-05-10 04:55:23,423 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-05-10 04:55:23,423 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-10 04:55:23,423 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-10 04:55:23,424 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,424 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-10 04:55:23,425 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,425 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,425 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:23,425 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,425 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,425 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,425 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:23,425 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,425 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:23,425 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:23,425 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:23,425 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:23,425 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,425 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:23,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:55:23,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:55:23,426 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:23,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:23,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:23,426 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:23,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:55:23,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,426 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:23,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,426 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:55:23,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:23,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:23,427 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:23,428 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:23,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:55:23,428 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:23,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,428 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:23,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:23,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:23,428 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:23,431 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:23,431 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,431 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:23,431 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:23,431 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:23,431 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:23,432 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-05-10 04:55:23,432 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-10 04:55:23,432 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:23,432 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:23,432 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:23,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:23,432 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:23,431 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:55:23,432 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-05-10 04:55:23,432 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-10 04:55:23,431 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:23,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:23,432 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:23,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:23,432 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:23,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:23,432 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:23,432 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,432 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:23,432 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,432 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:23,432 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:23,432 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:23,432 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:23,433 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,433 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,433 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:23,433 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:23,532 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-10 04:55:23,533 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-10 04:55:23,543 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-10 04:55:23,544 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-05-10 04:55:23,545 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,545 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-05-10 04:55:23,545 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:55:23,545 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-05-10 04:55:23,545 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-05-10 04:55:23,545 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,545 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,546 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:23,546 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:23,546 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:23,546 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:23,546 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:23,546 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:23,546 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:23,546 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:23,547 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:23,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:23,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:23,547 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-10 04:55:23,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,547 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,547 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,547 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:55:23,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,547 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:55:23,547 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:55:23,548 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,548 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,548 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,548 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-05-10 04:55:23,548 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,548 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-05-10 04:55:23,548 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-05-10 04:55:23,548 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-05-10 04:55:23,548 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:55:23,548 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,548 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-05-10 04:55:23,548 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-05-10 04:55:23,548 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,548 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37995,1683694419499' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-05-10 04:55:23,548 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-05-10 04:55:23,548 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:55:23,548 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-05-10 04:55:23,548 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:55:23,548 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-05-10 04:55:23,548 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-05-10 04:55:23,548 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-05-10 04:55:23,548 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37823,1683694419534' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-05-10 04:55:23,548 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-10 04:55:23,548 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-10 04:55:23,548 DEBUG [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1683694522515.4fb74b802ab843984094c342c1179292. 2023-05-10 04:55:23,549 DEBUG [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1683694522515.4fb74b802ab843984094c342c1179292. started... 2023-05-10 04:55:23,549 INFO [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 4fb74b802ab843984094c342c1179292 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-10 04:55:23,549 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,550 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,550 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:55:23,550 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,551 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,551 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:55:23,560 INFO [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/.tmp/cf1/5ded22c3ee7e4244bb0391ac0f834464 2023-05-10 04:55:23,570 INFO [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/.tmp/cf2/ef6d108aaad64c48a268da204d49a455 2023-05-10 04:55:23,573 DEBUG [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/.tmp/cf1/5ded22c3ee7e4244bb0391ac0f834464 as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/cf1/5ded22c3ee7e4244bb0391ac0f834464 2023-05-10 04:55:23,576 INFO [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/cf1/5ded22c3ee7e4244bb0391ac0f834464, entries=10, sequenceid=45, filesize=5.0 K 2023-05-10 04:55:23,577 DEBUG [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/.tmp/cf2/ef6d108aaad64c48a268da204d49a455 as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/cf2/ef6d108aaad64c48a268da204d49a455 2023-05-10 04:55:23,580 INFO [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/cf2/ef6d108aaad64c48a268da204d49a455, entries=10, sequenceid=45, filesize=5.0 K 2023-05-10 04:55:23,580 INFO [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 4fb74b802ab843984094c342c1179292 in 31ms, sequenceid=45, compaction requested=true 2023-05-10 04:55:23,581 DEBUG [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 4fb74b802ab843984094c342c1179292: 2023-05-10 04:55:23,581 DEBUG [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1683694522515.4fb74b802ab843984094c342c1179292. 2023-05-10 04:55:23,581 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-10 04:55:23,581 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-10 04:55:23,581 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:23,581 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-05-10 04:55:23,581 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36665,1683694419452' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-05-10 04:55:23,583 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:23,583 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:23,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:55:23,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:55:23,583 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:55:23,583 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:55:23,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:55:23,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:23,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:23,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:23,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:55:23,585 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36665,1683694419452' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-05-10 04:55:23,585 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2bf93623[Count = 0] remaining members to acquire global barrier 2023-05-10 04:55:23,585 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-05-10 04:55:23,585 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,586 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,586 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,586 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,586 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,586 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,586 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-05-10 04:55:23,587 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,587 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-05-10 04:55:23,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,587 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,37823,1683694419534' in zk 2023-05-10 04:55:23,587 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-05-10 04:55:23,587 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:23,587 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-10 04:55:23,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,587 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-05-10 04:55:23,587 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-05-10 04:55:23,587 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-05-10 04:55:23,587 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,37995,1683694419499' in zk 2023-05-10 04:55:23,587 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,36665,1683694419452' in zk 2023-05-10 04:55:23,588 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-05-10 04:55:23,588 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:55:23,588 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-05-10 04:55:23,589 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-05-10 04:55:23,589 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:55:23,589 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:23,589 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-05-10 04:55:23,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:23,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:55:23,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:55:23,589 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-05-10 04:55:23,589 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:55:23,589 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-05-10 04:55:23,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:55:23,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:55:23,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:23,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:23,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:23,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:55:23,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:23,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:23,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:23,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,36665,1683694419452': 2023-05-10 04:55:23,592 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36665,1683694419452' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-05-10 04:55:23,592 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-05-10 04:55:23,592 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-10 04:55:23,592 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-10 04:55:23,592 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,592 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-10 04:55:23,594 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,594 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,594 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,594 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,594 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:23,594 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,595 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,595 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:23,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:55:23,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:55:23,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,595 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:23,595 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,595 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:23,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:55:23,595 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:23,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,595 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:23,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:23,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:23,595 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:23,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,596 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:23,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:23,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:55:23,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,596 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:23,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:23,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:23,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:23,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:55:23,597 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:23,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,598 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:23,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:23,598 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:23,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:23,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:23,603 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:23,603 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,603 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:23,603 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:23,603 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:23,603 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:23,603 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:23,603 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:23,603 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:23,603 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,603 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:23,603 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:55:23,603 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-05-10 04:55:23,603 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-10 04:55:23,603 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:23,603 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:23,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:23,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:23,603 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:23,603 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:23,603 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:23,603 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-05-10 04:55:23,603 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,604 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:23,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:23,604 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-10 04:55:23,604 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:23,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:23,604 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:23,604 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:23,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:23,604 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:23,604 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,604 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,704 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-10 04:55:23,705 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-10 04:55:23,714 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-10 04:55:23,715 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-05-10 04:55:23,715 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,716 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-05-10 04:55:23,716 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:55:23,716 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-05-10 04:55:23,716 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-05-10 04:55:23,716 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,716 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,717 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:23,717 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:23,717 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:23,717 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:23,717 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:23,717 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:23,717 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:23,717 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:23,718 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-10 04:55:23,717 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:23,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:23,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:23,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,718 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,718 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:55:23,718 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,718 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,718 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:55:23,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,719 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,719 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-05-10 04:55:23,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:55:23,719 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,719 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,719 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-05-10 04:55:23,719 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,719 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-05-10 04:55:23,719 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:55:23,719 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,719 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-05-10 04:55:23,719 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-05-10 04:55:23,719 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-05-10 04:55:23,719 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:55:23,719 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,719 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-05-10 04:55:23,719 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-05-10 04:55:23,719 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-10 04:55:23,719 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-10 04:55:23,719 DEBUG [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1683694522515.4fb74b802ab843984094c342c1179292. 2023-05-10 04:55:23,719 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-05-10 04:55:23,719 DEBUG [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1683694522515.4fb74b802ab843984094c342c1179292. started... 2023-05-10 04:55:23,719 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37995,1683694419499' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-05-10 04:55:23,719 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:55:23,720 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-05-10 04:55:23,720 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-05-10 04:55:23,720 INFO [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 4fb74b802ab843984094c342c1179292 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-10 04:55:23,720 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37823,1683694419534' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-05-10 04:55:23,722 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,722 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,723 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:55:23,722 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,723 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,723 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:55:23,730 INFO [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/.tmp/cf1/670f40d0ebd64eb6aa77f0d8edc12d38 2023-05-10 04:55:23,742 INFO [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/.tmp/cf2/828e5b29242b42f687d74d496bc317ea 2023-05-10 04:55:23,745 DEBUG [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/.tmp/cf1/670f40d0ebd64eb6aa77f0d8edc12d38 as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/cf1/670f40d0ebd64eb6aa77f0d8edc12d38 2023-05-10 04:55:23,748 INFO [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/cf1/670f40d0ebd64eb6aa77f0d8edc12d38, entries=10, sequenceid=58, filesize=5.0 K 2023-05-10 04:55:23,748 DEBUG [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/.tmp/cf2/828e5b29242b42f687d74d496bc317ea as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/cf2/828e5b29242b42f687d74d496bc317ea 2023-05-10 04:55:23,751 INFO [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/cf2/828e5b29242b42f687d74d496bc317ea, entries=10, sequenceid=58, filesize=5.0 K 2023-05-10 04:55:23,752 INFO [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 4fb74b802ab843984094c342c1179292 in 32ms, sequenceid=58, compaction requested=true 2023-05-10 04:55:23,752 DEBUG [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 4fb74b802ab843984094c342c1179292: 2023-05-10 04:55:23,752 DEBUG [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1683694522515.4fb74b802ab843984094c342c1179292. 2023-05-10 04:55:23,752 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-10 04:55:23,752 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-10 04:55:23,752 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:23,752 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-05-10 04:55:23,752 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36665,1683694419452' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-05-10 04:55:23,754 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:23,754 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:23,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:55:23,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:55:23,754 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,754 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:55:23,754 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:55:23,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:55:23,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:23,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:23,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:23,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:55:23,756 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36665,1683694419452' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-05-10 04:55:23,756 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@307584af[Count = 0] remaining members to acquire global barrier 2023-05-10 04:55:23,756 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-05-10 04:55:23,756 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,757 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,757 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,757 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,757 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,757 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,757 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,757 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-05-10 04:55:23,757 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-05-10 04:55:23,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,757 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-05-10 04:55:23,757 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-05-10 04:55:23,757 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-05-10 04:55:23,757 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,37995,1683694419499' in zk 2023-05-10 04:55:23,757 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:23,757 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-10 04:55:23,757 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,37823,1683694419534' in zk 2023-05-10 04:55:23,757 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-05-10 04:55:23,758 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,36665,1683694419452' in zk 2023-05-10 04:55:23,760 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-05-10 04:55:23,760 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:55:23,760 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-05-10 04:55:23,760 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-05-10 04:55:23,761 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-05-10 04:55:23,761 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:55:23,761 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:55:23,761 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-05-10 04:55:23,761 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:23,761 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-05-10 04:55:23,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:23,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:55:23,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:55:23,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:55:23,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:55:23,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:23,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:23,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:23,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:55:23,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:23,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:23,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:23,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,36665,1683694419452': 2023-05-10 04:55:23,764 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36665,1683694419452' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-05-10 04:55:23,764 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-05-10 04:55:23,764 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-10 04:55:23,764 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-10 04:55:23,764 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,764 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-10 04:55:23,765 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,765 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,765 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:23,765 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,765 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:23,765 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,765 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,765 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,766 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:23,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:55:23,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:55:23,766 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:55:23,766 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:23,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:23,766 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:23,766 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:23,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:23,766 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:23,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:23,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,767 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:23,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:55:23,767 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:23,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:23,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:23,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:23,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:55:23,768 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:23,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,768 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:23,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:23,769 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:23,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:23,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:23,771 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:23,771 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:23,771 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:23,771 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:23,771 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:23,771 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:23,771 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:55:23,771 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,771 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-05-10 04:55:23,771 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-10 04:55:23,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:23,771 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:23,771 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:23,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:23,771 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:23,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:23,772 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:23,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:23,771 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:23,772 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-05-10 04:55:23,772 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:23,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:23,771 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:23,772 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-10 04:55:23,772 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:23,772 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:23,772 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:23,772 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,772 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:23,772 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,772 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:23,772 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:23,772 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,772 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,872 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-10 04:55:23,873 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-10 04:55:23,883 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-10 04:55:23,884 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-05-10 04:55:23,884 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,884 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-05-10 04:55:23,884 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:55:23,884 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-05-10 04:55:23,885 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-05-10 04:55:23,885 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,885 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,887 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:23,887 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:23,887 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:23,887 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:23,887 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:23,887 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:23,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:23,887 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:23,887 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-10 04:55:23,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:23,887 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:23,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:23,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,888 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,888 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,888 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:55:23,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:55:23,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,888 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,888 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:55:23,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,888 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,888 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-05-10 04:55:23,889 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,889 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-05-10 04:55:23,889 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-05-10 04:55:23,889 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,889 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:55:23,889 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-05-10 04:55:23,889 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-05-10 04:55:23,889 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-05-10 04:55:23,889 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37995,1683694419499' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-05-10 04:55:23,889 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-05-10 04:55:23,889 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:55:23,889 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,889 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-05-10 04:55:23,889 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-05-10 04:55:23,889 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:55:23,889 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-05-10 04:55:23,889 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37823,1683694419534' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-05-10 04:55:23,889 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-05-10 04:55:23,889 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-10 04:55:23,889 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-10 04:55:23,889 DEBUG [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1683694522515.4fb74b802ab843984094c342c1179292. 2023-05-10 04:55:23,889 DEBUG [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1683694522515.4fb74b802ab843984094c342c1179292. started... 2023-05-10 04:55:23,890 INFO [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 4fb74b802ab843984094c342c1179292 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-10 04:55:23,890 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,891 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,891 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,891 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:55:23,891 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,891 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:55:23,899 INFO [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/.tmp/cf1/322a8ffd0fc84cee9e0cf6c6c63bb8a1 2023-05-10 04:55:23,908 INFO [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/.tmp/cf2/99708bc70c3542b78d7a632ffffc82ad 2023-05-10 04:55:23,912 DEBUG [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/.tmp/cf1/322a8ffd0fc84cee9e0cf6c6c63bb8a1 as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/cf1/322a8ffd0fc84cee9e0cf6c6c63bb8a1 2023-05-10 04:55:23,915 INFO [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/cf1/322a8ffd0fc84cee9e0cf6c6c63bb8a1, entries=10, sequenceid=71, filesize=5.0 K 2023-05-10 04:55:23,915 DEBUG [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/.tmp/cf2/99708bc70c3542b78d7a632ffffc82ad as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/cf2/99708bc70c3542b78d7a632ffffc82ad 2023-05-10 04:55:23,918 INFO [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/cf2/99708bc70c3542b78d7a632ffffc82ad, entries=10, sequenceid=71, filesize=5.0 K 2023-05-10 04:55:23,919 INFO [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 4fb74b802ab843984094c342c1179292 in 30ms, sequenceid=71, compaction requested=true 2023-05-10 04:55:23,919 DEBUG [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 4fb74b802ab843984094c342c1179292: 2023-05-10 04:55:23,919 DEBUG [rs(jenkins-hbase4.apache.org,36665,1683694419452)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1683694522515.4fb74b802ab843984094c342c1179292. 2023-05-10 04:55:23,919 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-10 04:55:23,919 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-10 04:55:23,919 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:23,919 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-05-10 04:55:23,919 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36665,1683694419452' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-05-10 04:55:23,921 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:23,921 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:23,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:55:23,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:55:23,921 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,921 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:55:23,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:55:23,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:55:23,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:23,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:23,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:23,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:55:23,923 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,36665,1683694419452' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-05-10 04:55:23,923 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2f47b7dd[Count = 0] remaining members to acquire global barrier 2023-05-10 04:55:23,923 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-05-10 04:55:23,923 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,927 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,927 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,927 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,927 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,927 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:23,927 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,927 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-05-10 04:55:23,927 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,927 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-05-10 04:55:23,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,927 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-10 04:55:23,928 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-05-10 04:55:23,928 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-05-10 04:55:23,928 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,37995,1683694419499' in zk 2023-05-10 04:55:23,927 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,37823,1683694419534' in zk 2023-05-10 04:55:23,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,928 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-05-10 04:55:23,928 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-05-10 04:55:23,928 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase4.apache.org,36665,1683694419452' in zk 2023-05-10 04:55:23,929 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-05-10 04:55:23,929 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:55:23,929 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-05-10 04:55:23,930 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-05-10 04:55:23,930 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:23,930 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-05-10 04:55:23,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:23,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:55:23,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:55:23,930 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:55:23,930 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:55:23,930 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-05-10 04:55:23,930 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-05-10 04:55:23,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:55:23,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:55:23,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:23,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:23,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:23,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:55:23,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:23,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:23,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:23,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase4.apache.org,36665,1683694419452': 2023-05-10 04:55:23,933 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,36665,1683694419452' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-05-10 04:55:23,933 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-05-10 04:55:23,933 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-10 04:55:23,933 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-10 04:55:23,933 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,933 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-10 04:55:23,934 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,935 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,935 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,935 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,935 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,935 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:23,935 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,935 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:23,935 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:23,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:55:23,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:55:23,936 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:23,936 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:23,936 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:23,936 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:23,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:23,936 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:23,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:23,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,936 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:23,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:55:23,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,937 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:23,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:55:23,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:23,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:23,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:23,938 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:23,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:55:23,938 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:23,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,939 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:23,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:23,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:23,939 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:23,944 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:23,944 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:23,944 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:23,944 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,944 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:55:23,945 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:23,944 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:23,945 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:23,945 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:23,944 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-05-10 04:55:23,945 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-10 04:55:23,944 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:23,944 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:23,944 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:23,944 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:23,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:23,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:23,945 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,945 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:23,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:23,945 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:23,945 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-05-10 04:55:23,945 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:23,945 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:23,945 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:23,945 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:23,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:23,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:23,945 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-10 04:55:23,945 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:23,946 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:23,946 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:23,946 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-05-10 04:55:24,046 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-10 04:55:24,047 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-10 04:55:24,053 DEBUG [Listener at localhost/45007] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:24,054 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(1029): Stored pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=4fb74b802ab843984094c342c1179292, daughterA=8ce7399787d8eaa629fea6e9b52c42b1, daughterB=076ce7f4a590fa7fbbeb887426d15088 2023-05-10 04:55:24,055 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=4fb74b802ab843984094c342c1179292, daughterA=8ce7399787d8eaa629fea6e9b52c42b1, daughterB=076ce7f4a590fa7fbbeb887426d15088 2023-05-10 04:55:24,055 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=4fb74b802ab843984094c342c1179292, daughterA=8ce7399787d8eaa629fea6e9b52c42b1, daughterB=076ce7f4a590fa7fbbeb887426d15088 2023-05-10 04:55:24,055 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=4fb74b802ab843984094c342c1179292, daughterA=8ce7399787d8eaa629fea6e9b52c42b1, daughterB=076ce7f4a590fa7fbbeb887426d15088 2023-05-10 04:55:24,057 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=129, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=4fb74b802ab843984094c342c1179292, UNASSIGN}] 2023-05-10 04:55:24,057 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=129, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=4fb74b802ab843984094c342c1179292, UNASSIGN 2023-05-10 04:55:24,058 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=129 updating hbase:meta row=4fb74b802ab843984094c342c1179292, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:24,058 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1683694522515.4fb74b802ab843984094c342c1179292.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1683694524058"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683694524058"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683694524058"}]},"ts":"1683694524058"} 2023-05-10 04:55:24,059 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=130, ppid=129, state=RUNNABLE; CloseRegionProcedure 4fb74b802ab843984094c342c1179292, server=jenkins-hbase4.apache.org,36665,1683694419452}] 2023-05-10 04:55:24,211 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 4fb74b802ab843984094c342c1179292 2023-05-10 04:55:24,211 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 4fb74b802ab843984094c342c1179292, disabling compactions & flushes 2023-05-10 04:55:24,211 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionEmptyDaughter,,1683694522515.4fb74b802ab843984094c342c1179292. 2023-05-10 04:55:24,211 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,,1683694522515.4fb74b802ab843984094c342c1179292. 2023-05-10 04:55:24,211 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionEmptyDaughter,,1683694522515.4fb74b802ab843984094c342c1179292. after waiting 0 ms 2023-05-10 04:55:24,211 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionEmptyDaughter,,1683694522515.4fb74b802ab843984094c342c1179292. 2023-05-10 04:55:24,211 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 4fb74b802ab843984094c342c1179292 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-05-10 04:55:24,219 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/.tmp/cf1/28985eb29a154a17a651eaefd61179bb 2023-05-10 04:55:24,230 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/.tmp/cf2/b57fe5118a7043d2849ab70001a73e65 2023-05-10 04:55:24,233 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/.tmp/cf1/28985eb29a154a17a651eaefd61179bb as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/cf1/28985eb29a154a17a651eaefd61179bb 2023-05-10 04:55:24,237 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/cf1/28985eb29a154a17a651eaefd61179bb, entries=8, sequenceid=82, filesize=5.0 K 2023-05-10 04:55:24,237 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/.tmp/cf2/b57fe5118a7043d2849ab70001a73e65 as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/cf2/b57fe5118a7043d2849ab70001a73e65 2023-05-10 04:55:24,240 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/cf2/b57fe5118a7043d2849ab70001a73e65, entries=8, sequenceid=82, filesize=5.0 K 2023-05-10 04:55:24,241 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 4fb74b802ab843984094c342c1179292 in 30ms, sequenceid=82, compaction requested=true 2023-05-10 04:55:24,244 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-05-10 04:55:24,244 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-10 04:55:24,244 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionEmptyDaughter,,1683694522515.4fb74b802ab843984094c342c1179292. 2023-05-10 04:55:24,244 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 4fb74b802ab843984094c342c1179292: 2023-05-10 04:55:24,246 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 4fb74b802ab843984094c342c1179292 2023-05-10 04:55:24,246 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=129 updating hbase:meta row=4fb74b802ab843984094c342c1179292, regionState=CLOSED 2023-05-10 04:55:24,246 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,,1683694522515.4fb74b802ab843984094c342c1179292.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1683694524246"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683694524246"}]},"ts":"1683694524246"} 2023-05-10 04:55:24,248 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=130, resume processing ppid=129 2023-05-10 04:55:24,248 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=130, ppid=129, state=SUCCESS; CloseRegionProcedure 4fb74b802ab843984094c342c1179292, server=jenkins-hbase4.apache.org,36665,1683694419452 in 188 msec 2023-05-10 04:55:24,250 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=129, resume processing ppid=128 2023-05-10 04:55:24,250 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=129, ppid=128, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=4fb74b802ab843984094c342c1179292, UNASSIGN in 191 msec 2023-05-10 04:55:24,259 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=128 splitting 14 storefiles, region=4fb74b802ab843984094c342c1179292, threads=14 2023-05-10 04:55:24,259 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/cf2/1ebabb90982c4266a853080b40b2d69c for region: 4fb74b802ab843984094c342c1179292 2023-05-10 04:55:24,259 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/cf2/37ad6bf54a7f4109a4fa74e3d9607266 for region: 4fb74b802ab843984094c342c1179292 2023-05-10 04:55:24,259 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/cf2/828e5b29242b42f687d74d496bc317ea for region: 4fb74b802ab843984094c342c1179292 2023-05-10 04:55:24,259 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/cf2/8bfce04d38894ab1b6d10a9aef11f8a8 for region: 4fb74b802ab843984094c342c1179292 2023-05-10 04:55:24,259 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/cf2/99708bc70c3542b78d7a632ffffc82ad for region: 4fb74b802ab843984094c342c1179292 2023-05-10 04:55:24,262 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/cf2/b57fe5118a7043d2849ab70001a73e65 for region: 4fb74b802ab843984094c342c1179292 2023-05-10 04:55:24,263 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/cf2/ef6d108aaad64c48a268da204d49a455 for region: 4fb74b802ab843984094c342c1179292 2023-05-10 04:55:24,263 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/cf1/28985eb29a154a17a651eaefd61179bb for region: 4fb74b802ab843984094c342c1179292 2023-05-10 04:55:24,263 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/cf1/322a8ffd0fc84cee9e0cf6c6c63bb8a1 for region: 4fb74b802ab843984094c342c1179292 2023-05-10 04:55:24,266 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/cf1/5ded22c3ee7e4244bb0391ac0f834464 for region: 4fb74b802ab843984094c342c1179292 2023-05-10 04:55:24,267 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/cf1/670f40d0ebd64eb6aa77f0d8edc12d38 for region: 4fb74b802ab843984094c342c1179292 2023-05-10 04:55:24,267 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/cf1/ecc198964bd246519840a47b05401987 for region: 4fb74b802ab843984094c342c1179292 2023-05-10 04:55:24,270 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/cf1/f0e775b023894593afde5f0d4ff3f2e5 for region: 4fb74b802ab843984094c342c1179292 2023-05-10 04:55:24,271 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/cf1/fef4b1b4b5fe46a0bba604884b9694e3 for region: 4fb74b802ab843984094c342c1179292 2023-05-10 04:55:24,273 DEBUG [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/cf2/828e5b29242b42f687d74d496bc317ea, top=false 2023-05-10 04:55:24,273 DEBUG [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/cf2/1ebabb90982c4266a853080b40b2d69c, top=false 2023-05-10 04:55:24,274 DEBUG [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/cf2/37ad6bf54a7f4109a4fa74e3d9607266, top=false 2023-05-10 04:55:24,273 DEBUG [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/cf2/99708bc70c3542b78d7a632ffffc82ad, top=false 2023-05-10 04:55:24,281 DEBUG [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/cf1/5ded22c3ee7e4244bb0391ac0f834464, top=false 2023-05-10 04:55:24,282 DEBUG [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/cf1/28985eb29a154a17a651eaefd61179bb, top=false 2023-05-10 04:55:24,285 DEBUG [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/cf2/8bfce04d38894ab1b6d10a9aef11f8a8, top=false 2023-05-10 04:55:24,287 DEBUG [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/cf1/322a8ffd0fc84cee9e0cf6c6c63bb8a1, top=false 2023-05-10 04:55:24,290 DEBUG [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/cf2/ef6d108aaad64c48a268da204d49a455, top=false 2023-05-10 04:55:24,290 DEBUG [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/cf2/b57fe5118a7043d2849ab70001a73e65, top=false 2023-05-10 04:55:24,292 DEBUG [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/cf1/670f40d0ebd64eb6aa77f0d8edc12d38, top=false 2023-05-10 04:55:24,293 DEBUG [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/cf1/ecc198964bd246519840a47b05401987, top=false 2023-05-10 04:55:24,294 DEBUG [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/cf1/f0e775b023894593afde5f0d4ff3f2e5, top=false 2023-05-10 04:55:24,294 DEBUG [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/cf1/fef4b1b4b5fe46a0bba604884b9694e3, top=false 2023-05-10 04:55:24,297 INFO [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/.splits/8ce7399787d8eaa629fea6e9b52c42b1/cf2/testSplitTableRegionEmptyDaughter=4fb74b802ab843984094c342c1179292-37ad6bf54a7f4109a4fa74e3d9607266 for child: 8ce7399787d8eaa629fea6e9b52c42b1, parent: 4fb74b802ab843984094c342c1179292 2023-05-10 04:55:24,301 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/cf2/37ad6bf54a7f4109a4fa74e3d9607266 for region: 4fb74b802ab843984094c342c1179292 2023-05-10 04:55:24,307 INFO [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/.splits/8ce7399787d8eaa629fea6e9b52c42b1/cf2/testSplitTableRegionEmptyDaughter=4fb74b802ab843984094c342c1179292-99708bc70c3542b78d7a632ffffc82ad for child: 8ce7399787d8eaa629fea6e9b52c42b1, parent: 4fb74b802ab843984094c342c1179292 2023-05-10 04:55:24,307 INFO [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/.splits/8ce7399787d8eaa629fea6e9b52c42b1/cf2/testSplitTableRegionEmptyDaughter=4fb74b802ab843984094c342c1179292-828e5b29242b42f687d74d496bc317ea for child: 8ce7399787d8eaa629fea6e9b52c42b1, parent: 4fb74b802ab843984094c342c1179292 2023-05-10 04:55:24,310 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/cf2/99708bc70c3542b78d7a632ffffc82ad for region: 4fb74b802ab843984094c342c1179292 2023-05-10 04:55:24,310 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/cf2/828e5b29242b42f687d74d496bc317ea for region: 4fb74b802ab843984094c342c1179292 2023-05-10 04:55:24,315 INFO [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/.splits/8ce7399787d8eaa629fea6e9b52c42b1/cf1/testSplitTableRegionEmptyDaughter=4fb74b802ab843984094c342c1179292-f0e775b023894593afde5f0d4ff3f2e5 for child: 8ce7399787d8eaa629fea6e9b52c42b1, parent: 4fb74b802ab843984094c342c1179292 2023-05-10 04:55:24,317 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/cf1/f0e775b023894593afde5f0d4ff3f2e5 for region: 4fb74b802ab843984094c342c1179292 2023-05-10 04:55:24,318 INFO [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/.splits/8ce7399787d8eaa629fea6e9b52c42b1/cf2/testSplitTableRegionEmptyDaughter=4fb74b802ab843984094c342c1179292-b57fe5118a7043d2849ab70001a73e65 for child: 8ce7399787d8eaa629fea6e9b52c42b1, parent: 4fb74b802ab843984094c342c1179292 2023-05-10 04:55:24,318 INFO [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/.splits/8ce7399787d8eaa629fea6e9b52c42b1/cf2/testSplitTableRegionEmptyDaughter=4fb74b802ab843984094c342c1179292-ef6d108aaad64c48a268da204d49a455 for child: 8ce7399787d8eaa629fea6e9b52c42b1, parent: 4fb74b802ab843984094c342c1179292 2023-05-10 04:55:24,321 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/cf2/b57fe5118a7043d2849ab70001a73e65 for region: 4fb74b802ab843984094c342c1179292 2023-05-10 04:55:24,321 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/cf2/ef6d108aaad64c48a268da204d49a455 for region: 4fb74b802ab843984094c342c1179292 2023-05-10 04:55:24,323 INFO [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/.splits/8ce7399787d8eaa629fea6e9b52c42b1/cf1/testSplitTableRegionEmptyDaughter=4fb74b802ab843984094c342c1179292-ecc198964bd246519840a47b05401987 for child: 8ce7399787d8eaa629fea6e9b52c42b1, parent: 4fb74b802ab843984094c342c1179292 2023-05-10 04:55:24,327 INFO [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/.splits/8ce7399787d8eaa629fea6e9b52c42b1/cf1/testSplitTableRegionEmptyDaughter=4fb74b802ab843984094c342c1179292-fef4b1b4b5fe46a0bba604884b9694e3 for child: 8ce7399787d8eaa629fea6e9b52c42b1, parent: 4fb74b802ab843984094c342c1179292 2023-05-10 04:55:24,327 INFO [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/.splits/8ce7399787d8eaa629fea6e9b52c42b1/cf2/testSplitTableRegionEmptyDaughter=4fb74b802ab843984094c342c1179292-1ebabb90982c4266a853080b40b2d69c for child: 8ce7399787d8eaa629fea6e9b52c42b1, parent: 4fb74b802ab843984094c342c1179292 2023-05-10 04:55:24,327 INFO [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/.splits/8ce7399787d8eaa629fea6e9b52c42b1/cf2/testSplitTableRegionEmptyDaughter=4fb74b802ab843984094c342c1179292-8bfce04d38894ab1b6d10a9aef11f8a8 for child: 8ce7399787d8eaa629fea6e9b52c42b1, parent: 4fb74b802ab843984094c342c1179292 2023-05-10 04:55:24,328 INFO [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/.splits/8ce7399787d8eaa629fea6e9b52c42b1/cf1/testSplitTableRegionEmptyDaughter=4fb74b802ab843984094c342c1179292-322a8ffd0fc84cee9e0cf6c6c63bb8a1 for child: 8ce7399787d8eaa629fea6e9b52c42b1, parent: 4fb74b802ab843984094c342c1179292 2023-05-10 04:55:24,332 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/cf1/ecc198964bd246519840a47b05401987 for region: 4fb74b802ab843984094c342c1179292 2023-05-10 04:55:24,332 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/cf1/fef4b1b4b5fe46a0bba604884b9694e3 for region: 4fb74b802ab843984094c342c1179292 2023-05-10 04:55:24,332 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/cf2/1ebabb90982c4266a853080b40b2d69c for region: 4fb74b802ab843984094c342c1179292 2023-05-10 04:55:24,333 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/cf2/8bfce04d38894ab1b6d10a9aef11f8a8 for region: 4fb74b802ab843984094c342c1179292 2023-05-10 04:55:24,334 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/cf1/322a8ffd0fc84cee9e0cf6c6c63bb8a1 for region: 4fb74b802ab843984094c342c1179292 2023-05-10 04:55:24,334 INFO [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/.splits/8ce7399787d8eaa629fea6e9b52c42b1/cf1/testSplitTableRegionEmptyDaughter=4fb74b802ab843984094c342c1179292-5ded22c3ee7e4244bb0391ac0f834464 for child: 8ce7399787d8eaa629fea6e9b52c42b1, parent: 4fb74b802ab843984094c342c1179292 2023-05-10 04:55:24,334 INFO [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/.splits/8ce7399787d8eaa629fea6e9b52c42b1/cf1/testSplitTableRegionEmptyDaughter=4fb74b802ab843984094c342c1179292-670f40d0ebd64eb6aa77f0d8edc12d38 for child: 8ce7399787d8eaa629fea6e9b52c42b1, parent: 4fb74b802ab843984094c342c1179292 2023-05-10 04:55:24,334 INFO [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/.splits/8ce7399787d8eaa629fea6e9b52c42b1/cf1/testSplitTableRegionEmptyDaughter=4fb74b802ab843984094c342c1179292-28985eb29a154a17a651eaefd61179bb for child: 8ce7399787d8eaa629fea6e9b52c42b1, parent: 4fb74b802ab843984094c342c1179292 2023-05-10 04:55:24,337 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/cf1/5ded22c3ee7e4244bb0391ac0f834464 for region: 4fb74b802ab843984094c342c1179292 2023-05-10 04:55:24,337 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/cf1/670f40d0ebd64eb6aa77f0d8edc12d38 for region: 4fb74b802ab843984094c342c1179292 2023-05-10 04:55:24,337 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/cf1/28985eb29a154a17a651eaefd61179bb for region: 4fb74b802ab843984094c342c1179292 2023-05-10 04:55:24,338 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=128 split storefiles for region 4fb74b802ab843984094c342c1179292 Daughter A: 14 storefiles, Daughter B: 0 storefiles. 2023-05-10 04:55:24,366 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/8ce7399787d8eaa629fea6e9b52c42b1/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-05-10 04:55:24,367 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/076ce7f4a590fa7fbbeb887426d15088/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-05-10 04:55:24,369 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1683694522515.4fb74b802ab843984094c342c1179292.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1683694524369"},{"qualifier":"splitA","vlen":69,"tag":[],"timestamp":"1683694524369"},{"qualifier":"splitB","vlen":69,"tag":[],"timestamp":"1683694524369"}]},"ts":"1683694524369"} 2023-05-10 04:55:24,369 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1683694524053.8ce7399787d8eaa629fea6e9b52c42b1.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1683694524369"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683694524369"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683694524369"}]},"ts":"1683694524369"} 2023-05-10 04:55:24,369 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,71,1683694524053.076ce7f4a590fa7fbbeb887426d15088.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1683694524369"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683694524369"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683694524369"}]},"ts":"1683694524369"} 2023-05-10 04:55:24,372 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=131, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=8ce7399787d8eaa629fea6e9b52c42b1, ASSIGN}, {pid=132, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=076ce7f4a590fa7fbbeb887426d15088, ASSIGN}] 2023-05-10 04:55:24,372 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=132, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=076ce7f4a590fa7fbbeb887426d15088, ASSIGN 2023-05-10 04:55:24,375 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=132, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=076ce7f4a590fa7fbbeb887426d15088, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,36665,1683694419452; forceNewPlan=false, retain=false 2023-05-10 04:55:24,376 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=131, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=8ce7399787d8eaa629fea6e9b52c42b1, ASSIGN 2023-05-10 04:55:24,376 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=131, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=8ce7399787d8eaa629fea6e9b52c42b1, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,36665,1683694419452; forceNewPlan=false, retain=false 2023-05-10 04:55:24,525 INFO [jenkins-hbase4:37903] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-05-10 04:55:24,527 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=132 updating hbase:meta row=076ce7f4a590fa7fbbeb887426d15088, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:24,527 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,71,1683694524053.076ce7f4a590fa7fbbeb887426d15088.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1683694524527"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683694524527"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683694524527"}]},"ts":"1683694524527"} 2023-05-10 04:55:24,529 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=133, ppid=132, state=RUNNABLE; OpenRegionProcedure 076ce7f4a590fa7fbbeb887426d15088, server=jenkins-hbase4.apache.org,36665,1683694419452}] 2023-05-10 04:55:24,531 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=131 updating hbase:meta row=8ce7399787d8eaa629fea6e9b52c42b1, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:24,531 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1683694524053.8ce7399787d8eaa629fea6e9b52c42b1.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1683694524531"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683694524531"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683694524531"}]},"ts":"1683694524531"} 2023-05-10 04:55:24,532 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=134, ppid=131, state=RUNNABLE; OpenRegionProcedure 8ce7399787d8eaa629fea6e9b52c42b1, server=jenkins-hbase4.apache.org,36665,1683694419452}] 2023-05-10 04:55:24,683 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionEmptyDaughter,,1683694524053.8ce7399787d8eaa629fea6e9b52c42b1. 2023-05-10 04:55:24,683 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 8ce7399787d8eaa629fea6e9b52c42b1, NAME => 'testSplitTableRegionEmptyDaughter,,1683694524053.8ce7399787d8eaa629fea6e9b52c42b1.', STARTKEY => '', ENDKEY => '71'} 2023-05-10 04:55:24,683 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-05-10 04:55:24,684 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionEmptyDaughter 8ce7399787d8eaa629fea6e9b52c42b1 2023-05-10 04:55:24,684 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionEmptyDaughter,,1683694524053.8ce7399787d8eaa629fea6e9b52c42b1.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-10 04:55:24,684 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 8ce7399787d8eaa629fea6e9b52c42b1 2023-05-10 04:55:24,684 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 8ce7399787d8eaa629fea6e9b52c42b1 2023-05-10 04:55:24,685 INFO [StoreOpener-8ce7399787d8eaa629fea6e9b52c42b1-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 8ce7399787d8eaa629fea6e9b52c42b1 2023-05-10 04:55:24,685 DEBUG [StoreOpener-8ce7399787d8eaa629fea6e9b52c42b1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/8ce7399787d8eaa629fea6e9b52c42b1/cf1 2023-05-10 04:55:24,685 DEBUG [StoreOpener-8ce7399787d8eaa629fea6e9b52c42b1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/8ce7399787d8eaa629fea6e9b52c42b1/cf1 2023-05-10 04:55:24,686 INFO [StoreOpener-8ce7399787d8eaa629fea6e9b52c42b1-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 8ce7399787d8eaa629fea6e9b52c42b1 columnFamilyName cf1 2023-05-10 04:55:24,690 DEBUG [StoreOpener-8ce7399787d8eaa629fea6e9b52c42b1-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/8ce7399787d8eaa629fea6e9b52c42b1/cf1/testSplitTableRegionEmptyDaughter=4fb74b802ab843984094c342c1179292-28985eb29a154a17a651eaefd61179bb 2023-05-10 04:55:24,693 DEBUG [StoreOpener-8ce7399787d8eaa629fea6e9b52c42b1-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/8ce7399787d8eaa629fea6e9b52c42b1/cf1/testSplitTableRegionEmptyDaughter=4fb74b802ab843984094c342c1179292-322a8ffd0fc84cee9e0cf6c6c63bb8a1 2023-05-10 04:55:24,696 DEBUG [StoreOpener-8ce7399787d8eaa629fea6e9b52c42b1-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/8ce7399787d8eaa629fea6e9b52c42b1/cf1/testSplitTableRegionEmptyDaughter=4fb74b802ab843984094c342c1179292-5ded22c3ee7e4244bb0391ac0f834464 2023-05-10 04:55:24,698 DEBUG [StoreOpener-8ce7399787d8eaa629fea6e9b52c42b1-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/8ce7399787d8eaa629fea6e9b52c42b1/cf1/testSplitTableRegionEmptyDaughter=4fb74b802ab843984094c342c1179292-670f40d0ebd64eb6aa77f0d8edc12d38 2023-05-10 04:55:24,701 DEBUG [StoreOpener-8ce7399787d8eaa629fea6e9b52c42b1-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/8ce7399787d8eaa629fea6e9b52c42b1/cf1/testSplitTableRegionEmptyDaughter=4fb74b802ab843984094c342c1179292-ecc198964bd246519840a47b05401987 2023-05-10 04:55:24,704 DEBUG [StoreOpener-8ce7399787d8eaa629fea6e9b52c42b1-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/8ce7399787d8eaa629fea6e9b52c42b1/cf1/testSplitTableRegionEmptyDaughter=4fb74b802ab843984094c342c1179292-f0e775b023894593afde5f0d4ff3f2e5 2023-05-10 04:55:24,706 DEBUG [StoreOpener-8ce7399787d8eaa629fea6e9b52c42b1-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/8ce7399787d8eaa629fea6e9b52c42b1/cf1/testSplitTableRegionEmptyDaughter=4fb74b802ab843984094c342c1179292-fef4b1b4b5fe46a0bba604884b9694e3 2023-05-10 04:55:24,707 INFO [StoreOpener-8ce7399787d8eaa629fea6e9b52c42b1-1] regionserver.HStore(310): Store=8ce7399787d8eaa629fea6e9b52c42b1/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-10 04:55:24,707 INFO [StoreOpener-8ce7399787d8eaa629fea6e9b52c42b1-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 8ce7399787d8eaa629fea6e9b52c42b1 2023-05-10 04:55:24,707 DEBUG [StoreOpener-8ce7399787d8eaa629fea6e9b52c42b1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/8ce7399787d8eaa629fea6e9b52c42b1/cf2 2023-05-10 04:55:24,708 DEBUG [StoreOpener-8ce7399787d8eaa629fea6e9b52c42b1-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/8ce7399787d8eaa629fea6e9b52c42b1/cf2 2023-05-10 04:55:24,708 INFO [StoreOpener-8ce7399787d8eaa629fea6e9b52c42b1-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 8ce7399787d8eaa629fea6e9b52c42b1 columnFamilyName cf2 2023-05-10 04:55:24,712 DEBUG [StoreOpener-8ce7399787d8eaa629fea6e9b52c42b1-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/8ce7399787d8eaa629fea6e9b52c42b1/cf2/testSplitTableRegionEmptyDaughter=4fb74b802ab843984094c342c1179292-1ebabb90982c4266a853080b40b2d69c 2023-05-10 04:55:24,715 DEBUG [StoreOpener-8ce7399787d8eaa629fea6e9b52c42b1-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/8ce7399787d8eaa629fea6e9b52c42b1/cf2/testSplitTableRegionEmptyDaughter=4fb74b802ab843984094c342c1179292-37ad6bf54a7f4109a4fa74e3d9607266 2023-05-10 04:55:24,718 DEBUG [StoreOpener-8ce7399787d8eaa629fea6e9b52c42b1-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/8ce7399787d8eaa629fea6e9b52c42b1/cf2/testSplitTableRegionEmptyDaughter=4fb74b802ab843984094c342c1179292-828e5b29242b42f687d74d496bc317ea 2023-05-10 04:55:24,720 DEBUG [StoreOpener-8ce7399787d8eaa629fea6e9b52c42b1-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/8ce7399787d8eaa629fea6e9b52c42b1/cf2/testSplitTableRegionEmptyDaughter=4fb74b802ab843984094c342c1179292-8bfce04d38894ab1b6d10a9aef11f8a8 2023-05-10 04:55:24,723 DEBUG [StoreOpener-8ce7399787d8eaa629fea6e9b52c42b1-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/8ce7399787d8eaa629fea6e9b52c42b1/cf2/testSplitTableRegionEmptyDaughter=4fb74b802ab843984094c342c1179292-99708bc70c3542b78d7a632ffffc82ad 2023-05-10 04:55:24,725 DEBUG [StoreOpener-8ce7399787d8eaa629fea6e9b52c42b1-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/8ce7399787d8eaa629fea6e9b52c42b1/cf2/testSplitTableRegionEmptyDaughter=4fb74b802ab843984094c342c1179292-b57fe5118a7043d2849ab70001a73e65 2023-05-10 04:55:24,728 DEBUG [StoreOpener-8ce7399787d8eaa629fea6e9b52c42b1-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/8ce7399787d8eaa629fea6e9b52c42b1/cf2/testSplitTableRegionEmptyDaughter=4fb74b802ab843984094c342c1179292-ef6d108aaad64c48a268da204d49a455 2023-05-10 04:55:24,728 INFO [StoreOpener-8ce7399787d8eaa629fea6e9b52c42b1-1] regionserver.HStore(310): Store=8ce7399787d8eaa629fea6e9b52c42b1/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-10 04:55:24,729 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/8ce7399787d8eaa629fea6e9b52c42b1 2023-05-10 04:55:24,730 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/8ce7399787d8eaa629fea6e9b52c42b1 2023-05-10 04:55:24,731 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionEmptyDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-05-10 04:55:24,732 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 8ce7399787d8eaa629fea6e9b52c42b1 2023-05-10 04:55:24,733 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 8ce7399787d8eaa629fea6e9b52c42b1; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63615733, jitterRate=-0.05205170810222626}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-05-10 04:55:24,733 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 8ce7399787d8eaa629fea6e9b52c42b1: 2023-05-10 04:55:24,733 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2334): Post open deploy tasks for testSplitTableRegionEmptyDaughter,,1683694524053.8ce7399787d8eaa629fea6e9b52c42b1., pid=134, masterSystemTime=1683694524680 2023-05-10 04:55:24,733 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-10 04:55:24,734 DEBUG [RS:0;jenkins-hbase4:36665-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-05-10 04:55:24,734 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-05-10 04:55:24,734 DEBUG [RS:0;jenkins-hbase4:36665-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-05-10 04:55:24,735 DEBUG [RS:0;jenkins-hbase4:36665-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-05-10 04:55:24,735 DEBUG [RS:0;jenkins-hbase4:36665-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-05-10 04:55:24,735 DEBUG [RS:0;jenkins-hbase4:36665-shortCompactions-0] regionserver.HStore(1912): 8ce7399787d8eaa629fea6e9b52c42b1/cf1 is initiating minor compaction (all files) 2023-05-10 04:55:24,735 DEBUG [RS:0;jenkins-hbase4:36665-longCompactions-0] regionserver.HStore(1912): 8ce7399787d8eaa629fea6e9b52c42b1/cf2 is initiating minor compaction (all files) 2023-05-10 04:55:24,735 INFO [RS:0;jenkins-hbase4:36665-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 8ce7399787d8eaa629fea6e9b52c42b1/cf1 in testSplitTableRegionEmptyDaughter,,1683694524053.8ce7399787d8eaa629fea6e9b52c42b1. 2023-05-10 04:55:24,735 INFO [RS:0;jenkins-hbase4:36665-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 8ce7399787d8eaa629fea6e9b52c42b1/cf2 in testSplitTableRegionEmptyDaughter,,1683694524053.8ce7399787d8eaa629fea6e9b52c42b1. 2023-05-10 04:55:24,735 INFO [RS:0;jenkins-hbase4:36665-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/8ce7399787d8eaa629fea6e9b52c42b1/cf1/testSplitTableRegionEmptyDaughter=4fb74b802ab843984094c342c1179292-fef4b1b4b5fe46a0bba604884b9694e3, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/8ce7399787d8eaa629fea6e9b52c42b1/cf1/testSplitTableRegionEmptyDaughter=4fb74b802ab843984094c342c1179292-ecc198964bd246519840a47b05401987, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/8ce7399787d8eaa629fea6e9b52c42b1/cf1/testSplitTableRegionEmptyDaughter=4fb74b802ab843984094c342c1179292-f0e775b023894593afde5f0d4ff3f2e5, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/8ce7399787d8eaa629fea6e9b52c42b1/cf1/testSplitTableRegionEmptyDaughter=4fb74b802ab843984094c342c1179292-5ded22c3ee7e4244bb0391ac0f834464, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/8ce7399787d8eaa629fea6e9b52c42b1/cf1/testSplitTableRegionEmptyDaughter=4fb74b802ab843984094c342c1179292-670f40d0ebd64eb6aa77f0d8edc12d38, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/8ce7399787d8eaa629fea6e9b52c42b1/cf1/testSplitTableRegionEmptyDaughter=4fb74b802ab843984094c342c1179292-322a8ffd0fc84cee9e0cf6c6c63bb8a1, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/8ce7399787d8eaa629fea6e9b52c42b1/cf1/testSplitTableRegionEmptyDaughter=4fb74b802ab843984094c342c1179292-28985eb29a154a17a651eaefd61179bb] into tmpdir=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/8ce7399787d8eaa629fea6e9b52c42b1/.tmp, totalSize=34.9 K 2023-05-10 04:55:24,735 INFO [RS:0;jenkins-hbase4:36665-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/8ce7399787d8eaa629fea6e9b52c42b1/cf2/testSplitTableRegionEmptyDaughter=4fb74b802ab843984094c342c1179292-8bfce04d38894ab1b6d10a9aef11f8a8, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/8ce7399787d8eaa629fea6e9b52c42b1/cf2/testSplitTableRegionEmptyDaughter=4fb74b802ab843984094c342c1179292-37ad6bf54a7f4109a4fa74e3d9607266, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/8ce7399787d8eaa629fea6e9b52c42b1/cf2/testSplitTableRegionEmptyDaughter=4fb74b802ab843984094c342c1179292-1ebabb90982c4266a853080b40b2d69c, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/8ce7399787d8eaa629fea6e9b52c42b1/cf2/testSplitTableRegionEmptyDaughter=4fb74b802ab843984094c342c1179292-ef6d108aaad64c48a268da204d49a455, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/8ce7399787d8eaa629fea6e9b52c42b1/cf2/testSplitTableRegionEmptyDaughter=4fb74b802ab843984094c342c1179292-828e5b29242b42f687d74d496bc317ea, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/8ce7399787d8eaa629fea6e9b52c42b1/cf2/testSplitTableRegionEmptyDaughter=4fb74b802ab843984094c342c1179292-99708bc70c3542b78d7a632ffffc82ad, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/8ce7399787d8eaa629fea6e9b52c42b1/cf2/testSplitTableRegionEmptyDaughter=4fb74b802ab843984094c342c1179292-b57fe5118a7043d2849ab70001a73e65] into tmpdir=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/8ce7399787d8eaa629fea6e9b52c42b1/.tmp, totalSize=34.9 K 2023-05-10 04:55:24,735 DEBUG [RS:0;jenkins-hbase4:36665-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=4fb74b802ab843984094c342c1179292-fef4b1b4b5fe46a0bba604884b9694e3, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1683694523018 2023-05-10 04:55:24,735 DEBUG [RS:0;jenkins-hbase4:36665-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=4fb74b802ab843984094c342c1179292-8bfce04d38894ab1b6d10a9aef11f8a8, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1683694523018 2023-05-10 04:55:24,736 DEBUG [RS:0;jenkins-hbase4:36665-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=4fb74b802ab843984094c342c1179292-ecc198964bd246519840a47b05401987, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1683694523188 2023-05-10 04:55:24,736 DEBUG [RS:0;jenkins-hbase4:36665-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=4fb74b802ab843984094c342c1179292-37ad6bf54a7f4109a4fa74e3d9607266, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1683694523188 2023-05-10 04:55:24,736 DEBUG [RS:0;jenkins-hbase4:36665-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=4fb74b802ab843984094c342c1179292-f0e775b023894593afde5f0d4ff3f2e5, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1683694523353 2023-05-10 04:55:24,736 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2361): Finished post open deploy task for testSplitTableRegionEmptyDaughter,,1683694524053.8ce7399787d8eaa629fea6e9b52c42b1. 2023-05-10 04:55:24,736 DEBUG [RS:0;jenkins-hbase4:36665-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=4fb74b802ab843984094c342c1179292-1ebabb90982c4266a853080b40b2d69c, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1683694523353 2023-05-10 04:55:24,736 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionEmptyDaughter,,1683694524053.8ce7399787d8eaa629fea6e9b52c42b1. 2023-05-10 04:55:24,736 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionEmptyDaughter,71,1683694524053.076ce7f4a590fa7fbbeb887426d15088. 2023-05-10 04:55:24,737 DEBUG [RS:0;jenkins-hbase4:36665-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=4fb74b802ab843984094c342c1179292-5ded22c3ee7e4244bb0391ac0f834464, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1683694523534 2023-05-10 04:55:24,737 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 076ce7f4a590fa7fbbeb887426d15088, NAME => 'testSplitTableRegionEmptyDaughter,71,1683694524053.076ce7f4a590fa7fbbeb887426d15088.', STARTKEY => '71', ENDKEY => ''} 2023-05-10 04:55:24,737 DEBUG [RS:0;jenkins-hbase4:36665-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=4fb74b802ab843984094c342c1179292-ef6d108aaad64c48a268da204d49a455, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1683694523534 2023-05-10 04:55:24,737 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=131 updating hbase:meta row=8ce7399787d8eaa629fea6e9b52c42b1, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:24,737 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-05-10 04:55:24,737 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionEmptyDaughter,,1683694524053.8ce7399787d8eaa629fea6e9b52c42b1.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1683694524737"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1683694524737"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1683694524737"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683694524737"}]},"ts":"1683694524737"} 2023-05-10 04:55:24,737 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionEmptyDaughter 076ce7f4a590fa7fbbeb887426d15088 2023-05-10 04:55:24,737 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionEmptyDaughter,71,1683694524053.076ce7f4a590fa7fbbeb887426d15088.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-10 04:55:24,737 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 076ce7f4a590fa7fbbeb887426d15088 2023-05-10 04:55:24,737 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 076ce7f4a590fa7fbbeb887426d15088 2023-05-10 04:55:24,737 DEBUG [RS:0;jenkins-hbase4:36665-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=4fb74b802ab843984094c342c1179292-670f40d0ebd64eb6aa77f0d8edc12d38, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1683694523705 2023-05-10 04:55:24,738 DEBUG [RS:0;jenkins-hbase4:36665-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=4fb74b802ab843984094c342c1179292-828e5b29242b42f687d74d496bc317ea, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1683694523705 2023-05-10 04:55:24,738 DEBUG [RS:0;jenkins-hbase4:36665-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=4fb74b802ab843984094c342c1179292-322a8ffd0fc84cee9e0cf6c6c63bb8a1, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1683694523874 2023-05-10 04:55:24,739 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-05-10 04:55:24,740 DEBUG [RS:0;jenkins-hbase4:36665-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=4fb74b802ab843984094c342c1179292-99708bc70c3542b78d7a632ffffc82ad, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1683694523874 2023-05-10 04:55:24,740 DEBUG [RS:0;jenkins-hbase4:36665-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=4fb74b802ab843984094c342c1179292-28985eb29a154a17a651eaefd61179bb, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1683694524047 2023-05-10 04:55:24,741 DEBUG [RS:0;jenkins-hbase4:36665-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=4fb74b802ab843984094c342c1179292-b57fe5118a7043d2849ab70001a73e65, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1683694524047 2023-05-10 04:55:24,743 INFO [StoreOpener-076ce7f4a590fa7fbbeb887426d15088-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 076ce7f4a590fa7fbbeb887426d15088 2023-05-10 04:55:24,743 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=134, resume processing ppid=131 2023-05-10 04:55:24,744 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=134, ppid=131, state=SUCCESS; OpenRegionProcedure 8ce7399787d8eaa629fea6e9b52c42b1, server=jenkins-hbase4.apache.org,36665,1683694419452 in 207 msec 2023-05-10 04:55:24,745 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=131, ppid=128, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=8ce7399787d8eaa629fea6e9b52c42b1, ASSIGN in 373 msec 2023-05-10 04:55:24,747 DEBUG [StoreOpener-076ce7f4a590fa7fbbeb887426d15088-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/076ce7f4a590fa7fbbeb887426d15088/cf1 2023-05-10 04:55:24,747 DEBUG [StoreOpener-076ce7f4a590fa7fbbeb887426d15088-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/076ce7f4a590fa7fbbeb887426d15088/cf1 2023-05-10 04:55:24,748 INFO [StoreOpener-076ce7f4a590fa7fbbeb887426d15088-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 076ce7f4a590fa7fbbeb887426d15088 columnFamilyName cf1 2023-05-10 04:55:24,748 INFO [StoreOpener-076ce7f4a590fa7fbbeb887426d15088-1] regionserver.HStore(310): Store=076ce7f4a590fa7fbbeb887426d15088/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-10 04:55:24,748 INFO [StoreOpener-076ce7f4a590fa7fbbeb887426d15088-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 076ce7f4a590fa7fbbeb887426d15088 2023-05-10 04:55:24,750 DEBUG [StoreOpener-076ce7f4a590fa7fbbeb887426d15088-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/076ce7f4a590fa7fbbeb887426d15088/cf2 2023-05-10 04:55:24,750 DEBUG [StoreOpener-076ce7f4a590fa7fbbeb887426d15088-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/076ce7f4a590fa7fbbeb887426d15088/cf2 2023-05-10 04:55:24,750 INFO [StoreOpener-076ce7f4a590fa7fbbeb887426d15088-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 076ce7f4a590fa7fbbeb887426d15088 columnFamilyName cf2 2023-05-10 04:55:24,751 INFO [StoreOpener-076ce7f4a590fa7fbbeb887426d15088-1] regionserver.HStore(310): Store=076ce7f4a590fa7fbbeb887426d15088/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-10 04:55:24,752 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/076ce7f4a590fa7fbbeb887426d15088 2023-05-10 04:55:24,754 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/076ce7f4a590fa7fbbeb887426d15088 2023-05-10 04:55:24,758 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionEmptyDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-05-10 04:55:24,760 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 076ce7f4a590fa7fbbeb887426d15088 2023-05-10 04:55:24,761 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 076ce7f4a590fa7fbbeb887426d15088; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65108563, jitterRate=-0.02980680763721466}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-05-10 04:55:24,761 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 076ce7f4a590fa7fbbeb887426d15088: 2023-05-10 04:55:24,762 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2334): Post open deploy tasks for testSplitTableRegionEmptyDaughter,71,1683694524053.076ce7f4a590fa7fbbeb887426d15088., pid=133, masterSystemTime=1683694524680 2023-05-10 04:55:24,764 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2361): Finished post open deploy task for testSplitTableRegionEmptyDaughter,71,1683694524053.076ce7f4a590fa7fbbeb887426d15088. 2023-05-10 04:55:24,764 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionEmptyDaughter,71,1683694524053.076ce7f4a590fa7fbbeb887426d15088. 2023-05-10 04:55:24,764 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=132 updating hbase:meta row=076ce7f4a590fa7fbbeb887426d15088, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:24,764 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionEmptyDaughter,71,1683694524053.076ce7f4a590fa7fbbeb887426d15088.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1683694524764"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1683694524764"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1683694524764"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683694524764"}]},"ts":"1683694524764"} 2023-05-10 04:55:24,767 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=133, resume processing ppid=132 2023-05-10 04:55:24,767 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=133, ppid=132, state=SUCCESS; OpenRegionProcedure 076ce7f4a590fa7fbbeb887426d15088, server=jenkins-hbase4.apache.org,36665,1683694419452 in 236 msec 2023-05-10 04:55:24,769 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=132, resume processing ppid=128 2023-05-10 04:55:24,769 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=132, ppid=128, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=076ce7f4a590fa7fbbeb887426d15088, ASSIGN in 396 msec 2023-05-10 04:55:24,771 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=128, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=4fb74b802ab843984094c342c1179292, daughterA=8ce7399787d8eaa629fea6e9b52c42b1, daughterB=076ce7f4a590fa7fbbeb887426d15088 in 717 msec 2023-05-10 04:55:24,780 INFO [RS:0;jenkins-hbase4:36665-longCompactions-0] throttle.PressureAwareThroughputController(145): 8ce7399787d8eaa629fea6e9b52c42b1#cf2#compaction#124 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-05-10 04:55:24,787 INFO [RS:0;jenkins-hbase4:36665-shortCompactions-0] throttle.PressureAwareThroughputController(145): 8ce7399787d8eaa629fea6e9b52c42b1#cf1#compaction#125 average throughput is 1.72 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-05-10 04:55:24,807 DEBUG [RS:0;jenkins-hbase4:36665-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/8ce7399787d8eaa629fea6e9b52c42b1/.tmp/cf2/307010f3a6e34c7ca1cbfc1f958250a1 as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/8ce7399787d8eaa629fea6e9b52c42b1/cf2/307010f3a6e34c7ca1cbfc1f958250a1 2023-05-10 04:55:24,811 INFO [RS:0;jenkins-hbase4:36665-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 8ce7399787d8eaa629fea6e9b52c42b1/cf2 of 8ce7399787d8eaa629fea6e9b52c42b1 into 307010f3a6e34c7ca1cbfc1f958250a1(size=7.3 K), total size for store is 7.3 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-05-10 04:55:24,811 DEBUG [RS:0;jenkins-hbase4:36665-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 8ce7399787d8eaa629fea6e9b52c42b1: 2023-05-10 04:55:24,811 INFO [RS:0;jenkins-hbase4:36665-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionEmptyDaughter,,1683694524053.8ce7399787d8eaa629fea6e9b52c42b1., storeName=8ce7399787d8eaa629fea6e9b52c42b1/cf2, priority=9, startTime=1683694524734; duration=0sec 2023-05-10 04:55:24,815 DEBUG [RS:0;jenkins-hbase4:36665-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-10 04:55:24,816 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36665] ipc.CallRunner(144): callId: 1019 service: ClientService methodName: Scan size: 157 connection: 172.31.14.131:54092 deadline: 1683694584815, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitTableRegionEmptyDaughter,,1683694522515.4fb74b802ab843984094c342c1179292. is not online on jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:24,822 DEBUG [RS:0;jenkins-hbase4:36665-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/8ce7399787d8eaa629fea6e9b52c42b1/.tmp/cf1/169520e874494998aa82af2bc62fa21a as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/8ce7399787d8eaa629fea6e9b52c42b1/cf1/169520e874494998aa82af2bc62fa21a 2023-05-10 04:55:24,826 INFO [RS:0;jenkins-hbase4:36665-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 8ce7399787d8eaa629fea6e9b52c42b1/cf1 of 8ce7399787d8eaa629fea6e9b52c42b1 into 169520e874494998aa82af2bc62fa21a(size=7.3 K), total size for store is 7.3 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-05-10 04:55:24,826 DEBUG [RS:0;jenkins-hbase4:36665-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 8ce7399787d8eaa629fea6e9b52c42b1: 2023-05-10 04:55:24,826 INFO [RS:0;jenkins-hbase4:36665-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionEmptyDaughter,,1683694524053.8ce7399787d8eaa629fea6e9b52c42b1., storeName=8ce7399787d8eaa629fea6e9b52c42b1/cf1, priority=9, startTime=1683694524733; duration=0sec 2023-05-10 04:55:24,826 DEBUG [RS:0;jenkins-hbase4:36665-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-10 04:55:24,926 DEBUG [Listener at localhost/45007] client.ClientScanner(240): Advancing internal scanner to startKey at '71', inclusive 2023-05-10 04:55:24,929 WARN [Listener at localhost/45007] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-05-10 04:55:24,930 INFO [Listener at localhost/45007] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionEmptyDaughter 2023-05-10 04:55:24,931 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testSplitTableRegionEmptyDaughter 2023-05-10 04:55:24,932 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] procedure2.ProcedureExecutor(1029): Stored pid=135, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionEmptyDaughter 2023-05-10 04:55:24,934 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(1227): Checking to see if procedure is done pid=135 2023-05-10 04:55:24,935 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683694524935"}]},"ts":"1683694524935"} 2023-05-10 04:55:24,935 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=DISABLING in hbase:meta 2023-05-10 04:55:24,937 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionEmptyDaughter to state=DISABLING 2023-05-10 04:55:24,938 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=136, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=8ce7399787d8eaa629fea6e9b52c42b1, UNASSIGN}, {pid=137, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=076ce7f4a590fa7fbbeb887426d15088, UNASSIGN}] 2023-05-10 04:55:24,938 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=137, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=076ce7f4a590fa7fbbeb887426d15088, UNASSIGN 2023-05-10 04:55:24,939 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=137 updating hbase:meta row=076ce7f4a590fa7fbbeb887426d15088, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:24,939 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,71,1683694524053.076ce7f4a590fa7fbbeb887426d15088.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1683694524939"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683694524939"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683694524939"}]},"ts":"1683694524939"} 2023-05-10 04:55:24,940 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=138, ppid=137, state=RUNNABLE; CloseRegionProcedure 076ce7f4a590fa7fbbeb887426d15088, server=jenkins-hbase4.apache.org,36665,1683694419452}] 2023-05-10 04:55:24,942 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=136, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=8ce7399787d8eaa629fea6e9b52c42b1, UNASSIGN 2023-05-10 04:55:24,942 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=136 updating hbase:meta row=8ce7399787d8eaa629fea6e9b52c42b1, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:24,943 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1683694524053.8ce7399787d8eaa629fea6e9b52c42b1.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1683694524942"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683694524942"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683694524942"}]},"ts":"1683694524942"} 2023-05-10 04:55:24,943 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=139, ppid=136, state=RUNNABLE; CloseRegionProcedure 8ce7399787d8eaa629fea6e9b52c42b1, server=jenkins-hbase4.apache.org,36665,1683694419452}] 2023-05-10 04:55:25,035 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(1227): Checking to see if procedure is done pid=135 2023-05-10 04:55:25,067 ERROR [BP-726857689-172.31.14.131-1683694414559 heartbeating to localhost/127.0.0.1:38753] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-726857689-172.31.14.131-1683694414559 (Datanode Uuid a7b70f5c-571f-439d-b854-0d142c7d1170) service to localhost/127.0.0.1:38753 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-05-10 04:55:25,092 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 8ce7399787d8eaa629fea6e9b52c42b1 2023-05-10 04:55:25,092 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 8ce7399787d8eaa629fea6e9b52c42b1, disabling compactions & flushes 2023-05-10 04:55:25,092 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionEmptyDaughter,,1683694524053.8ce7399787d8eaa629fea6e9b52c42b1. 2023-05-10 04:55:25,092 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,,1683694524053.8ce7399787d8eaa629fea6e9b52c42b1. 2023-05-10 04:55:25,092 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionEmptyDaughter,,1683694524053.8ce7399787d8eaa629fea6e9b52c42b1. after waiting 0 ms 2023-05-10 04:55:25,092 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionEmptyDaughter,,1683694524053.8ce7399787d8eaa629fea6e9b52c42b1. 2023-05-10 04:55:25,093 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1683694524053.8ce7399787d8eaa629fea6e9b52c42b1.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/8ce7399787d8eaa629fea6e9b52c42b1/cf1/testSplitTableRegionEmptyDaughter=4fb74b802ab843984094c342c1179292-fef4b1b4b5fe46a0bba604884b9694e3, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/8ce7399787d8eaa629fea6e9b52c42b1/cf1/testSplitTableRegionEmptyDaughter=4fb74b802ab843984094c342c1179292-ecc198964bd246519840a47b05401987, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/8ce7399787d8eaa629fea6e9b52c42b1/cf1/testSplitTableRegionEmptyDaughter=4fb74b802ab843984094c342c1179292-f0e775b023894593afde5f0d4ff3f2e5, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/8ce7399787d8eaa629fea6e9b52c42b1/cf1/testSplitTableRegionEmptyDaughter=4fb74b802ab843984094c342c1179292-5ded22c3ee7e4244bb0391ac0f834464, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/8ce7399787d8eaa629fea6e9b52c42b1/cf1/testSplitTableRegionEmptyDaughter=4fb74b802ab843984094c342c1179292-670f40d0ebd64eb6aa77f0d8edc12d38, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/8ce7399787d8eaa629fea6e9b52c42b1/cf1/testSplitTableRegionEmptyDaughter=4fb74b802ab843984094c342c1179292-322a8ffd0fc84cee9e0cf6c6c63bb8a1, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/8ce7399787d8eaa629fea6e9b52c42b1/cf1/testSplitTableRegionEmptyDaughter=4fb74b802ab843984094c342c1179292-28985eb29a154a17a651eaefd61179bb] to archive 2023-05-10 04:55:25,094 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1683694524053.8ce7399787d8eaa629fea6e9b52c42b1.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-05-10 04:55:25,096 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1683694524053.8ce7399787d8eaa629fea6e9b52c42b1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/8ce7399787d8eaa629fea6e9b52c42b1/cf1/testSplitTableRegionEmptyDaughter=4fb74b802ab843984094c342c1179292-fef4b1b4b5fe46a0bba604884b9694e3 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionEmptyDaughter/8ce7399787d8eaa629fea6e9b52c42b1/cf1/testSplitTableRegionEmptyDaughter=4fb74b802ab843984094c342c1179292-fef4b1b4b5fe46a0bba604884b9694e3 2023-05-10 04:55:25,097 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1683694524053.8ce7399787d8eaa629fea6e9b52c42b1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/8ce7399787d8eaa629fea6e9b52c42b1/cf1/testSplitTableRegionEmptyDaughter=4fb74b802ab843984094c342c1179292-ecc198964bd246519840a47b05401987 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionEmptyDaughter/8ce7399787d8eaa629fea6e9b52c42b1/cf1/testSplitTableRegionEmptyDaughter=4fb74b802ab843984094c342c1179292-ecc198964bd246519840a47b05401987 2023-05-10 04:55:25,098 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1683694524053.8ce7399787d8eaa629fea6e9b52c42b1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/8ce7399787d8eaa629fea6e9b52c42b1/cf1/testSplitTableRegionEmptyDaughter=4fb74b802ab843984094c342c1179292-f0e775b023894593afde5f0d4ff3f2e5 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionEmptyDaughter/8ce7399787d8eaa629fea6e9b52c42b1/cf1/testSplitTableRegionEmptyDaughter=4fb74b802ab843984094c342c1179292-f0e775b023894593afde5f0d4ff3f2e5 2023-05-10 04:55:25,099 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1683694524053.8ce7399787d8eaa629fea6e9b52c42b1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/8ce7399787d8eaa629fea6e9b52c42b1/cf1/testSplitTableRegionEmptyDaughter=4fb74b802ab843984094c342c1179292-5ded22c3ee7e4244bb0391ac0f834464 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionEmptyDaughter/8ce7399787d8eaa629fea6e9b52c42b1/cf1/testSplitTableRegionEmptyDaughter=4fb74b802ab843984094c342c1179292-5ded22c3ee7e4244bb0391ac0f834464 2023-05-10 04:55:25,100 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1683694524053.8ce7399787d8eaa629fea6e9b52c42b1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/8ce7399787d8eaa629fea6e9b52c42b1/cf1/testSplitTableRegionEmptyDaughter=4fb74b802ab843984094c342c1179292-670f40d0ebd64eb6aa77f0d8edc12d38 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionEmptyDaughter/8ce7399787d8eaa629fea6e9b52c42b1/cf1/testSplitTableRegionEmptyDaughter=4fb74b802ab843984094c342c1179292-670f40d0ebd64eb6aa77f0d8edc12d38 2023-05-10 04:55:25,101 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1683694524053.8ce7399787d8eaa629fea6e9b52c42b1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/8ce7399787d8eaa629fea6e9b52c42b1/cf1/testSplitTableRegionEmptyDaughter=4fb74b802ab843984094c342c1179292-322a8ffd0fc84cee9e0cf6c6c63bb8a1 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionEmptyDaughter/8ce7399787d8eaa629fea6e9b52c42b1/cf1/testSplitTableRegionEmptyDaughter=4fb74b802ab843984094c342c1179292-322a8ffd0fc84cee9e0cf6c6c63bb8a1 2023-05-10 04:55:25,102 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1683694524053.8ce7399787d8eaa629fea6e9b52c42b1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/8ce7399787d8eaa629fea6e9b52c42b1/cf1/testSplitTableRegionEmptyDaughter=4fb74b802ab843984094c342c1179292-28985eb29a154a17a651eaefd61179bb to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionEmptyDaughter/8ce7399787d8eaa629fea6e9b52c42b1/cf1/testSplitTableRegionEmptyDaughter=4fb74b802ab843984094c342c1179292-28985eb29a154a17a651eaefd61179bb 2023-05-10 04:55:25,104 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1683694524053.8ce7399787d8eaa629fea6e9b52c42b1.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/8ce7399787d8eaa629fea6e9b52c42b1/cf2/testSplitTableRegionEmptyDaughter=4fb74b802ab843984094c342c1179292-8bfce04d38894ab1b6d10a9aef11f8a8, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/8ce7399787d8eaa629fea6e9b52c42b1/cf2/testSplitTableRegionEmptyDaughter=4fb74b802ab843984094c342c1179292-37ad6bf54a7f4109a4fa74e3d9607266, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/8ce7399787d8eaa629fea6e9b52c42b1/cf2/testSplitTableRegionEmptyDaughter=4fb74b802ab843984094c342c1179292-1ebabb90982c4266a853080b40b2d69c, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/8ce7399787d8eaa629fea6e9b52c42b1/cf2/testSplitTableRegionEmptyDaughter=4fb74b802ab843984094c342c1179292-ef6d108aaad64c48a268da204d49a455, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/8ce7399787d8eaa629fea6e9b52c42b1/cf2/testSplitTableRegionEmptyDaughter=4fb74b802ab843984094c342c1179292-828e5b29242b42f687d74d496bc317ea, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/8ce7399787d8eaa629fea6e9b52c42b1/cf2/testSplitTableRegionEmptyDaughter=4fb74b802ab843984094c342c1179292-99708bc70c3542b78d7a632ffffc82ad, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/8ce7399787d8eaa629fea6e9b52c42b1/cf2/testSplitTableRegionEmptyDaughter=4fb74b802ab843984094c342c1179292-b57fe5118a7043d2849ab70001a73e65] to archive 2023-05-10 04:55:25,104 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1683694524053.8ce7399787d8eaa629fea6e9b52c42b1.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-05-10 04:55:25,105 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1683694524053.8ce7399787d8eaa629fea6e9b52c42b1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/8ce7399787d8eaa629fea6e9b52c42b1/cf2/testSplitTableRegionEmptyDaughter=4fb74b802ab843984094c342c1179292-8bfce04d38894ab1b6d10a9aef11f8a8 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionEmptyDaughter/8ce7399787d8eaa629fea6e9b52c42b1/cf2/testSplitTableRegionEmptyDaughter=4fb74b802ab843984094c342c1179292-8bfce04d38894ab1b6d10a9aef11f8a8 2023-05-10 04:55:25,106 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1683694524053.8ce7399787d8eaa629fea6e9b52c42b1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/8ce7399787d8eaa629fea6e9b52c42b1/cf2/testSplitTableRegionEmptyDaughter=4fb74b802ab843984094c342c1179292-37ad6bf54a7f4109a4fa74e3d9607266 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionEmptyDaughter/8ce7399787d8eaa629fea6e9b52c42b1/cf2/testSplitTableRegionEmptyDaughter=4fb74b802ab843984094c342c1179292-37ad6bf54a7f4109a4fa74e3d9607266 2023-05-10 04:55:25,107 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1683694524053.8ce7399787d8eaa629fea6e9b52c42b1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/8ce7399787d8eaa629fea6e9b52c42b1/cf2/testSplitTableRegionEmptyDaughter=4fb74b802ab843984094c342c1179292-1ebabb90982c4266a853080b40b2d69c to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionEmptyDaughter/8ce7399787d8eaa629fea6e9b52c42b1/cf2/testSplitTableRegionEmptyDaughter=4fb74b802ab843984094c342c1179292-1ebabb90982c4266a853080b40b2d69c 2023-05-10 04:55:25,108 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1683694524053.8ce7399787d8eaa629fea6e9b52c42b1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/8ce7399787d8eaa629fea6e9b52c42b1/cf2/testSplitTableRegionEmptyDaughter=4fb74b802ab843984094c342c1179292-ef6d108aaad64c48a268da204d49a455 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionEmptyDaughter/8ce7399787d8eaa629fea6e9b52c42b1/cf2/testSplitTableRegionEmptyDaughter=4fb74b802ab843984094c342c1179292-ef6d108aaad64c48a268da204d49a455 2023-05-10 04:55:25,109 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1683694524053.8ce7399787d8eaa629fea6e9b52c42b1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/8ce7399787d8eaa629fea6e9b52c42b1/cf2/testSplitTableRegionEmptyDaughter=4fb74b802ab843984094c342c1179292-828e5b29242b42f687d74d496bc317ea to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionEmptyDaughter/8ce7399787d8eaa629fea6e9b52c42b1/cf2/testSplitTableRegionEmptyDaughter=4fb74b802ab843984094c342c1179292-828e5b29242b42f687d74d496bc317ea 2023-05-10 04:55:25,110 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1683694524053.8ce7399787d8eaa629fea6e9b52c42b1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/8ce7399787d8eaa629fea6e9b52c42b1/cf2/testSplitTableRegionEmptyDaughter=4fb74b802ab843984094c342c1179292-99708bc70c3542b78d7a632ffffc82ad to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionEmptyDaughter/8ce7399787d8eaa629fea6e9b52c42b1/cf2/testSplitTableRegionEmptyDaughter=4fb74b802ab843984094c342c1179292-99708bc70c3542b78d7a632ffffc82ad 2023-05-10 04:55:25,111 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1683694524053.8ce7399787d8eaa629fea6e9b52c42b1.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/8ce7399787d8eaa629fea6e9b52c42b1/cf2/testSplitTableRegionEmptyDaughter=4fb74b802ab843984094c342c1179292-b57fe5118a7043d2849ab70001a73e65 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionEmptyDaughter/8ce7399787d8eaa629fea6e9b52c42b1/cf2/testSplitTableRegionEmptyDaughter=4fb74b802ab843984094c342c1179292-b57fe5118a7043d2849ab70001a73e65 2023-05-10 04:55:25,114 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/8ce7399787d8eaa629fea6e9b52c42b1/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-05-10 04:55:25,114 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-10 04:55:25,115 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionEmptyDaughter,,1683694524053.8ce7399787d8eaa629fea6e9b52c42b1. 2023-05-10 04:55:25,115 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 8ce7399787d8eaa629fea6e9b52c42b1: 2023-05-10 04:55:25,117 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 8ce7399787d8eaa629fea6e9b52c42b1 2023-05-10 04:55:25,117 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 076ce7f4a590fa7fbbeb887426d15088 2023-05-10 04:55:25,118 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 076ce7f4a590fa7fbbeb887426d15088, disabling compactions & flushes 2023-05-10 04:55:25,118 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionEmptyDaughter,71,1683694524053.076ce7f4a590fa7fbbeb887426d15088. 2023-05-10 04:55:25,118 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,71,1683694524053.076ce7f4a590fa7fbbeb887426d15088. 2023-05-10 04:55:25,118 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionEmptyDaughter,71,1683694524053.076ce7f4a590fa7fbbeb887426d15088. after waiting 0 ms 2023-05-10 04:55:25,118 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionEmptyDaughter,71,1683694524053.076ce7f4a590fa7fbbeb887426d15088. 2023-05-10 04:55:25,118 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=136 updating hbase:meta row=8ce7399787d8eaa629fea6e9b52c42b1, regionState=CLOSED 2023-05-10 04:55:25,118 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,,1683694524053.8ce7399787d8eaa629fea6e9b52c42b1.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1683694525118"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683694525118"}]},"ts":"1683694525118"} 2023-05-10 04:55:25,120 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegionEmptyDaughter/076ce7f4a590fa7fbbeb887426d15088/recovered.edits/88.seqid, newMaxSeqId=88, maxSeqId=85 2023-05-10 04:55:25,121 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-10 04:55:25,122 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionEmptyDaughter,71,1683694524053.076ce7f4a590fa7fbbeb887426d15088. 2023-05-10 04:55:25,122 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 076ce7f4a590fa7fbbeb887426d15088: 2023-05-10 04:55:25,123 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 076ce7f4a590fa7fbbeb887426d15088 2023-05-10 04:55:25,123 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=139, resume processing ppid=136 2023-05-10 04:55:25,124 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=139, ppid=136, state=SUCCESS; CloseRegionProcedure 8ce7399787d8eaa629fea6e9b52c42b1, server=jenkins-hbase4.apache.org,36665,1683694419452 in 179 msec 2023-05-10 04:55:25,125 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=136, ppid=135, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=8ce7399787d8eaa629fea6e9b52c42b1, UNASSIGN in 186 msec 2023-05-10 04:55:25,125 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=137 updating hbase:meta row=076ce7f4a590fa7fbbeb887426d15088, regionState=CLOSED 2023-05-10 04:55:25,125 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,71,1683694524053.076ce7f4a590fa7fbbeb887426d15088.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1683694525125"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683694525125"}]},"ts":"1683694525125"} 2023-05-10 04:55:25,128 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=138, resume processing ppid=137 2023-05-10 04:55:25,128 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=138, ppid=137, state=SUCCESS; CloseRegionProcedure 076ce7f4a590fa7fbbeb887426d15088, server=jenkins-hbase4.apache.org,36665,1683694419452 in 186 msec 2023-05-10 04:55:25,129 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=137, resume processing ppid=135 2023-05-10 04:55:25,129 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=137, ppid=135, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=076ce7f4a590fa7fbbeb887426d15088, UNASSIGN in 190 msec 2023-05-10 04:55:25,129 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683694525129"}]},"ts":"1683694525129"} 2023-05-10 04:55:25,130 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=DISABLED in hbase:meta 2023-05-10 04:55:25,131 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionEmptyDaughter to state=DISABLED 2023-05-10 04:55:25,133 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=135, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionEmptyDaughter in 201 msec 2023-05-10 04:55:25,236 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(1227): Checking to see if procedure is done pid=135 2023-05-10 04:55:25,237 INFO [Listener at localhost/45007] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionEmptyDaughter, procId: 135 completed 2023-05-10 04:55:25,237 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete testSplitTableRegionEmptyDaughter 2023-05-10 04:55:25,238 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] procedure2.ProcedureExecutor(1029): Stored pid=140, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-05-10 04:55:25,239 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=140, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-05-10 04:55:25,240 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=140, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-05-10 04:55:25,240 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(1227): Checking to see if procedure is done pid=140 2023-05-10 04:55:25,244 DEBUG [HFileArchiver-9] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292 2023-05-10 04:55:25,244 DEBUG [HFileArchiver-10] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionEmptyDaughter/8ce7399787d8eaa629fea6e9b52c42b1 2023-05-10 04:55:25,244 DEBUG [HFileArchiver-11] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionEmptyDaughter/076ce7f4a590fa7fbbeb887426d15088 2023-05-10 04:55:25,246 DEBUG [HFileArchiver-9] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/cf1, FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/cf2, FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/recovered.edits] 2023-05-10 04:55:25,246 DEBUG [HFileArchiver-11] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionEmptyDaughter/076ce7f4a590fa7fbbeb887426d15088/cf1, FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionEmptyDaughter/076ce7f4a590fa7fbbeb887426d15088/cf2, FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionEmptyDaughter/076ce7f4a590fa7fbbeb887426d15088/recovered.edits] 2023-05-10 04:55:25,246 DEBUG [HFileArchiver-10] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionEmptyDaughter/8ce7399787d8eaa629fea6e9b52c42b1/cf1, FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionEmptyDaughter/8ce7399787d8eaa629fea6e9b52c42b1/cf2, FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionEmptyDaughter/8ce7399787d8eaa629fea6e9b52c42b1/recovered.edits] 2023-05-10 04:55:25,248 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/cf1/28985eb29a154a17a651eaefd61179bb to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/cf1/28985eb29a154a17a651eaefd61179bb 2023-05-10 04:55:25,248 DEBUG [HFileArchiver-10] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionEmptyDaughter/8ce7399787d8eaa629fea6e9b52c42b1/cf1/169520e874494998aa82af2bc62fa21a to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionEmptyDaughter/8ce7399787d8eaa629fea6e9b52c42b1/cf1/169520e874494998aa82af2bc62fa21a 2023-05-10 04:55:25,250 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/cf1/322a8ffd0fc84cee9e0cf6c6c63bb8a1 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/cf1/322a8ffd0fc84cee9e0cf6c6c63bb8a1 2023-05-10 04:55:25,251 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionEmptyDaughter/076ce7f4a590fa7fbbeb887426d15088/recovered.edits/88.seqid to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionEmptyDaughter/076ce7f4a590fa7fbbeb887426d15088/recovered.edits/88.seqid 2023-05-10 04:55:25,251 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/cf1/5ded22c3ee7e4244bb0391ac0f834464 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/cf1/5ded22c3ee7e4244bb0391ac0f834464 2023-05-10 04:55:25,251 DEBUG [HFileArchiver-10] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionEmptyDaughter/8ce7399787d8eaa629fea6e9b52c42b1/cf2/307010f3a6e34c7ca1cbfc1f958250a1 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionEmptyDaughter/8ce7399787d8eaa629fea6e9b52c42b1/cf2/307010f3a6e34c7ca1cbfc1f958250a1 2023-05-10 04:55:25,252 DEBUG [HFileArchiver-11] backup.HFileArchiver(596): Deleted hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionEmptyDaughter/076ce7f4a590fa7fbbeb887426d15088 2023-05-10 04:55:25,253 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/cf1/670f40d0ebd64eb6aa77f0d8edc12d38 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/cf1/670f40d0ebd64eb6aa77f0d8edc12d38 2023-05-10 04:55:25,254 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/cf1/ecc198964bd246519840a47b05401987 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/cf1/ecc198964bd246519840a47b05401987 2023-05-10 04:55:25,254 DEBUG [HFileArchiver-10] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionEmptyDaughter/8ce7399787d8eaa629fea6e9b52c42b1/recovered.edits/90.seqid to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionEmptyDaughter/8ce7399787d8eaa629fea6e9b52c42b1/recovered.edits/90.seqid 2023-05-10 04:55:25,255 DEBUG [HFileArchiver-10] backup.HFileArchiver(596): Deleted hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionEmptyDaughter/8ce7399787d8eaa629fea6e9b52c42b1 2023-05-10 04:55:25,255 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/cf1/f0e775b023894593afde5f0d4ff3f2e5 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/cf1/f0e775b023894593afde5f0d4ff3f2e5 2023-05-10 04:55:25,256 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/cf1/fef4b1b4b5fe46a0bba604884b9694e3 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/cf1/fef4b1b4b5fe46a0bba604884b9694e3 2023-05-10 04:55:25,258 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/cf2/1ebabb90982c4266a853080b40b2d69c to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/cf2/1ebabb90982c4266a853080b40b2d69c 2023-05-10 04:55:25,259 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/cf2/37ad6bf54a7f4109a4fa74e3d9607266 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/cf2/37ad6bf54a7f4109a4fa74e3d9607266 2023-05-10 04:55:25,260 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/cf2/828e5b29242b42f687d74d496bc317ea to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/cf2/828e5b29242b42f687d74d496bc317ea 2023-05-10 04:55:25,261 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/cf2/8bfce04d38894ab1b6d10a9aef11f8a8 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/cf2/8bfce04d38894ab1b6d10a9aef11f8a8 2023-05-10 04:55:25,262 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/cf2/99708bc70c3542b78d7a632ffffc82ad to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/cf2/99708bc70c3542b78d7a632ffffc82ad 2023-05-10 04:55:25,263 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/cf2/b57fe5118a7043d2849ab70001a73e65 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/cf2/b57fe5118a7043d2849ab70001a73e65 2023-05-10 04:55:25,264 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/cf2/ef6d108aaad64c48a268da204d49a455 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/cf2/ef6d108aaad64c48a268da204d49a455 2023-05-10 04:55:25,266 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/recovered.edits/85.seqid to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292/recovered.edits/85.seqid 2023-05-10 04:55:25,266 DEBUG [HFileArchiver-9] backup.HFileArchiver(596): Deleted hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegionEmptyDaughter/4fb74b802ab843984094c342c1179292 2023-05-10 04:55:25,266 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionEmptyDaughter regions 2023-05-10 04:55:25,268 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=140, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-05-10 04:55:25,269 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionEmptyDaughter from hbase:meta 2023-05-10 04:55:25,271 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionEmptyDaughter' descriptor. 2023-05-10 04:55:25,271 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=140, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-05-10 04:55:25,271 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionEmptyDaughter' from region states. 2023-05-10 04:55:25,272 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter,,1683694522515.4fb74b802ab843984094c342c1179292.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1683694525271"}]},"ts":"9223372036854775807"} 2023-05-10 04:55:25,272 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter,,1683694524053.8ce7399787d8eaa629fea6e9b52c42b1.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1683694525271"}]},"ts":"9223372036854775807"} 2023-05-10 04:55:25,272 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter,71,1683694524053.076ce7f4a590fa7fbbeb887426d15088.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1683694525271"}]},"ts":"9223372036854775807"} 2023-05-10 04:55:25,273 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-05-10 04:55:25,273 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 4fb74b802ab843984094c342c1179292, NAME => 'testSplitTableRegionEmptyDaughter,,1683694522515.4fb74b802ab843984094c342c1179292.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 8ce7399787d8eaa629fea6e9b52c42b1, NAME => 'testSplitTableRegionEmptyDaughter,,1683694524053.8ce7399787d8eaa629fea6e9b52c42b1.', STARTKEY => '', ENDKEY => '71'}, {ENCODED => 076ce7f4a590fa7fbbeb887426d15088, NAME => 'testSplitTableRegionEmptyDaughter,71,1683694524053.076ce7f4a590fa7fbbeb887426d15088.', STARTKEY => '71', ENDKEY => ''}] 2023-05-10 04:55:25,273 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionEmptyDaughter' as deleted. 2023-05-10 04:55:25,273 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1683694525273"}]},"ts":"9223372036854775807"} 2023-05-10 04:55:25,274 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionEmptyDaughter state from META 2023-05-10 04:55:25,276 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=140, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-05-10 04:55:25,277 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=140, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter in 39 msec 2023-05-10 04:55:25,341 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(1227): Checking to see if procedure is done pid=140 2023-05-10 04:55:25,342 INFO [Listener at localhost/45007] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionEmptyDaughter, procId: 140 completed 2023-05-10 04:55:25,357 INFO [Listener at localhost/45007] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionEmptyDaughter Thread=377 (was 377), OpenFileDescriptor=835 (was 835), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=173 (was 171) - SystemLoadAverage LEAK? -, ProcessCount=174 (was 174), AvailableMemoryMB=9532 (was 9537) 2023-05-10 04:55:25,372 INFO [Listener at localhost/45007] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegion Thread=377, OpenFileDescriptor=835, MaxFileDescriptor=60000, SystemLoadAverage=173, ProcessCount=174, AvailableMemoryMB=9531 2023-05-10 04:55:25,373 WARN [Listener at localhost/45007] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-05-10 04:55:25,375 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-05-10 04:55:25,376 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(492): Client=jenkins//172.31.14.131 set balanceSwitch=false 2023-05-10 04:55:25,377 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(1029): Stored pid=141, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegion 2023-05-10 04:55:25,379 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_PRE_OPERATION 2023-05-10 04:55:25,379 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-05-10 04:55:25,380 DEBUG [HFileArchiver-12] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600 2023-05-10 04:55:25,381 DEBUG [HFileArchiver-12] backup.HFileArchiver(153): Directory hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600 empty. 2023-05-10 04:55:25,381 DEBUG [HFileArchiver-12] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600 2023-05-10 04:55:25,381 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegion regions 2023-05-10 04:55:25,390 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegion/.tabledesc/.tableinfo.0000000001 2023-05-10 04:55:25,391 INFO [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(7675): creating {ENCODED => e9be1c226b4184298a60d5e1076cd600, NAME => 'testSplitTableRegion,,1683694525376.e9be1c226b4184298a60d5e1076cd600.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegion', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp 2023-05-10 04:55:25,398 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(866): Instantiated testSplitTableRegion,,1683694525376.e9be1c226b4184298a60d5e1076cd600.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-10 04:55:25,398 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1604): Closing e9be1c226b4184298a60d5e1076cd600, disabling compactions & flushes 2023-05-10 04:55:25,398 INFO [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1626): Closing region testSplitTableRegion,,1683694525376.e9be1c226b4184298a60d5e1076cd600. 2023-05-10 04:55:25,398 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegion,,1683694525376.e9be1c226b4184298a60d5e1076cd600. 2023-05-10 04:55:25,398 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegion,,1683694525376.e9be1c226b4184298a60d5e1076cd600. after waiting 0 ms 2023-05-10 04:55:25,398 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegion,,1683694525376.e9be1c226b4184298a60d5e1076cd600. 2023-05-10 04:55:25,398 INFO [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1838): Closed testSplitTableRegion,,1683694525376.e9be1c226b4184298a60d5e1076cd600. 2023-05-10 04:55:25,398 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1558): Region close journal for e9be1c226b4184298a60d5e1076cd600: 2023-05-10 04:55:25,400 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_ADD_TO_META 2023-05-10 04:55:25,400 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegion,,1683694525376.e9be1c226b4184298a60d5e1076cd600.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1683694525400"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683694525400"}]},"ts":"1683694525400"} 2023-05-10 04:55:25,401 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-05-10 04:55:25,402 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-05-10 04:55:25,402 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683694525402"}]},"ts":"1683694525402"} 2023-05-10 04:55:25,403 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=ENABLING in hbase:meta 2023-05-10 04:55:25,405 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase4.apache.org=0} racks are {/default-rack=0} 2023-05-10 04:55:25,406 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-05-10 04:55:25,406 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-05-10 04:55:25,406 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-05-10 04:55:25,406 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-05-10 04:55:25,406 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=142, ppid=141, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=e9be1c226b4184298a60d5e1076cd600, ASSIGN}] 2023-05-10 04:55:25,407 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=142, ppid=141, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=e9be1c226b4184298a60d5e1076cd600, ASSIGN 2023-05-10 04:55:25,408 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=142, ppid=141, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegion, region=e9be1c226b4184298a60d5e1076cd600, ASSIGN; state=OFFLINE, location=jenkins-hbase4.apache.org,37823,1683694419534; forceNewPlan=false, retain=false 2023-05-10 04:55:25,558 INFO [jenkins-hbase4:37903] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-05-10 04:55:25,559 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=142 updating hbase:meta row=e9be1c226b4184298a60d5e1076cd600, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:25,559 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1683694525376.e9be1c226b4184298a60d5e1076cd600.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1683694525559"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683694525559"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683694525559"}]},"ts":"1683694525559"} 2023-05-10 04:55:25,561 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=143, ppid=142, state=RUNNABLE; OpenRegionProcedure e9be1c226b4184298a60d5e1076cd600, server=jenkins-hbase4.apache.org,37823,1683694419534}] 2023-05-10 04:55:25,740 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegion,,1683694525376.e9be1c226b4184298a60d5e1076cd600. 2023-05-10 04:55:25,740 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => e9be1c226b4184298a60d5e1076cd600, NAME => 'testSplitTableRegion,,1683694525376.e9be1c226b4184298a60d5e1076cd600.', STARTKEY => '', ENDKEY => ''} 2023-05-10 04:55:25,740 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-05-10 04:55:25,740 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegion e9be1c226b4184298a60d5e1076cd600 2023-05-10 04:55:25,740 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegion,,1683694525376.e9be1c226b4184298a60d5e1076cd600.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-10 04:55:25,741 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for e9be1c226b4184298a60d5e1076cd600 2023-05-10 04:55:25,741 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for e9be1c226b4184298a60d5e1076cd600 2023-05-10 04:55:25,742 INFO [StoreOpener-e9be1c226b4184298a60d5e1076cd600-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region e9be1c226b4184298a60d5e1076cd600 2023-05-10 04:55:25,743 DEBUG [StoreOpener-e9be1c226b4184298a60d5e1076cd600-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf1 2023-05-10 04:55:25,743 DEBUG [StoreOpener-e9be1c226b4184298a60d5e1076cd600-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf1 2023-05-10 04:55:25,743 INFO [StoreOpener-e9be1c226b4184298a60d5e1076cd600-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e9be1c226b4184298a60d5e1076cd600 columnFamilyName cf1 2023-05-10 04:55:25,743 INFO [StoreOpener-e9be1c226b4184298a60d5e1076cd600-1] regionserver.HStore(310): Store=e9be1c226b4184298a60d5e1076cd600/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-10 04:55:25,744 INFO [StoreOpener-e9be1c226b4184298a60d5e1076cd600-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region e9be1c226b4184298a60d5e1076cd600 2023-05-10 04:55:25,744 DEBUG [StoreOpener-e9be1c226b4184298a60d5e1076cd600-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf2 2023-05-10 04:55:25,745 DEBUG [StoreOpener-e9be1c226b4184298a60d5e1076cd600-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf2 2023-05-10 04:55:25,745 INFO [StoreOpener-e9be1c226b4184298a60d5e1076cd600-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e9be1c226b4184298a60d5e1076cd600 columnFamilyName cf2 2023-05-10 04:55:25,745 INFO [StoreOpener-e9be1c226b4184298a60d5e1076cd600-1] regionserver.HStore(310): Store=e9be1c226b4184298a60d5e1076cd600/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-10 04:55:25,746 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600 2023-05-10 04:55:25,746 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600 2023-05-10 04:55:25,747 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegion descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-05-10 04:55:25,748 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for e9be1c226b4184298a60d5e1076cd600 2023-05-10 04:55:25,750 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-05-10 04:55:25,750 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened e9be1c226b4184298a60d5e1076cd600; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63617696, jitterRate=-0.052022457122802734}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-05-10 04:55:25,750 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for e9be1c226b4184298a60d5e1076cd600: 2023-05-10 04:55:25,751 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2334): Post open deploy tasks for testSplitTableRegion,,1683694525376.e9be1c226b4184298a60d5e1076cd600., pid=143, masterSystemTime=1683694525712 2023-05-10 04:55:25,752 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2361): Finished post open deploy task for testSplitTableRegion,,1683694525376.e9be1c226b4184298a60d5e1076cd600. 2023-05-10 04:55:25,752 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegion,,1683694525376.e9be1c226b4184298a60d5e1076cd600. 2023-05-10 04:55:25,752 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=142 updating hbase:meta row=e9be1c226b4184298a60d5e1076cd600, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:25,753 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegion,,1683694525376.e9be1c226b4184298a60d5e1076cd600.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1683694525752"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1683694525752"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1683694525752"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683694525752"}]},"ts":"1683694525752"} 2023-05-10 04:55:25,755 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=143, resume processing ppid=142 2023-05-10 04:55:25,755 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=143, ppid=142, state=SUCCESS; OpenRegionProcedure e9be1c226b4184298a60d5e1076cd600, server=jenkins-hbase4.apache.org,37823,1683694419534 in 193 msec 2023-05-10 04:55:25,756 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=142, resume processing ppid=141 2023-05-10 04:55:25,756 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=142, ppid=141, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=e9be1c226b4184298a60d5e1076cd600, ASSIGN in 349 msec 2023-05-10 04:55:25,757 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-05-10 04:55:25,757 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683694525757"}]},"ts":"1683694525757"} 2023-05-10 04:55:25,758 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=ENABLED in hbase:meta 2023-05-10 04:55:25,760 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_POST_OPERATION 2023-05-10 04:55:25,760 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=141, state=SUCCESS; CreateTableProcedure table=testSplitTableRegion in 384 msec 2023-05-10 04:55:25,884 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-10 04:55:25,885 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-05-10 04:55:25,885 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-05-10 04:55:25,885 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:55:25,885 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-05-10 04:55:25,885 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-05-10 04:55:25,886 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-10 04:55:25,886 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-10 04:55:25,888 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:25,888 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:25,888 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:25,888 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:25,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:25,888 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:25,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:25,888 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:25,888 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:25,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:25,888 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:25,888 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-10 04:55:25,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-10 04:55:25,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-10 04:55:25,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-10 04:55:25,889 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-10 04:55:25,889 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-10 04:55:25,889 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-10 04:55:25,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:55:25,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-10 04:55:25,889 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-05-10 04:55:25,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:55:25,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-10 04:55:25,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:55:25,889 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-10 04:55:25,889 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-05-10 04:55:25,889 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-05-10 04:55:25,889 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-05-10 04:55:25,889 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-05-10 04:55:25,889 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:55:25,889 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-05-10 04:55:25,890 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-05-10 04:55:25,890 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-05-10 04:55:25,890 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:55:25,890 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-05-10 04:55:25,890 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-05-10 04:55:25,890 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-05-10 04:55:25,890 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-05-10 04:55:25,890 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36665,1683694419452' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-05-10 04:55:25,890 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-05-10 04:55:25,890 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37995,1683694419499' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-05-10 04:55:25,890 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:55:25,890 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-05-10 04:55:25,890 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-10 04:55:25,890 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-10 04:55:25,890 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1683694525376.e9be1c226b4184298a60d5e1076cd600. 2023-05-10 04:55:25,890 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1683694525376.e9be1c226b4184298a60d5e1076cd600. started... 2023-05-10 04:55:25,891 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing e9be1c226b4184298a60d5e1076cd600 2/2 column families, dataSize=120 B heapSize=928 B 2023-05-10 04:55:25,892 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-10 04:55:25,893 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-10 04:55:25,893 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-10 04:55:25,893 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:55:25,893 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-10 04:55:25,893 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:55:25,903 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/.tmp/cf1/58d433c9ed254876ac1bfa1b33856169 2023-05-10 04:55:25,914 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/.tmp/cf2/4664a731a30c46e2bf2e8bad410a89bd 2023-05-10 04:55:25,917 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/.tmp/cf1/58d433c9ed254876ac1bfa1b33856169 as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf1/58d433c9ed254876ac1bfa1b33856169 2023-05-10 04:55:25,920 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf1/58d433c9ed254876ac1bfa1b33856169, entries=2, sequenceid=6, filesize=4.8 K 2023-05-10 04:55:25,920 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/.tmp/cf2/4664a731a30c46e2bf2e8bad410a89bd as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf2/4664a731a30c46e2bf2e8bad410a89bd 2023-05-10 04:55:25,923 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf2/4664a731a30c46e2bf2e8bad410a89bd, entries=2, sequenceid=6, filesize=4.8 K 2023-05-10 04:55:25,924 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for e9be1c226b4184298a60d5e1076cd600 in 33ms, sequenceid=6, compaction requested=false 2023-05-10 04:55:25,924 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegion' 2023-05-10 04:55:25,925 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for e9be1c226b4184298a60d5e1076cd600: 2023-05-10 04:55:25,925 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1683694525376.e9be1c226b4184298a60d5e1076cd600. 2023-05-10 04:55:25,925 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-10 04:55:25,925 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-10 04:55:25,925 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:25,925 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-05-10 04:55:25,925 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37823,1683694419534' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-05-10 04:55:25,926 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:25,926 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-10 04:55:25,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:25,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:55:25,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:55:25,926 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-10 04:55:25,926 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:55:25,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:55:25,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:55:25,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-10 04:55:25,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:25,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:25,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:25,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:55:25,928 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37823,1683694419534' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-05-10 04:55:25,928 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@b779d74[Count = 0] remaining members to acquire global barrier 2023-05-10 04:55:25,928 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-05-10 04:55:25,928 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-10 04:55:25,930 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-10 04:55:25,930 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-10 04:55:25,930 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-10 04:55:25,930 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-10 04:55:25,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-10 04:55:25,930 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-10 04:55:25,930 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:25,930 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-10 04:55:25,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-10 04:55:25,930 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-05-10 04:55:25,930 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-05-10 04:55:25,931 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-05-10 04:55:25,930 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-10 04:55:25,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-10 04:55:25,931 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,37823,1683694419534' in zk 2023-05-10 04:55:25,931 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-05-10 04:55:25,931 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-05-10 04:55:25,930 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-05-10 04:55:25,931 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,37995,1683694419499' in zk 2023-05-10 04:55:25,931 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,36665,1683694419452' in zk 2023-05-10 04:55:25,932 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:25,932 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-05-10 04:55:25,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:25,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:55:25,932 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:55:25,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:55:25,932 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-05-10 04:55:25,935 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-05-10 04:55:25,935 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-05-10 04:55:25,935 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:55:25,935 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:55:25,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:55:25,935 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-05-10 04:55:25,935 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-05-10 04:55:25,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:55:25,935 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-10 04:55:25,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:25,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:25,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:25,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:55:25,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-10 04:55:25,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:25,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:25,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:25,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,37823,1683694419534': 2023-05-10 04:55:25,937 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37823,1683694419534' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-05-10 04:55:25,937 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-05-10 04:55:25,938 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-10 04:55:25,938 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-10 04:55:25,938 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-05-10 04:55:25,938 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-10 04:55:25,941 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-10 04:55:25,941 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-10 04:55:25,941 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-10 04:55:25,941 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-10 04:55:25,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-10 04:55:25,941 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-10 04:55:25,941 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:25,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-10 04:55:25,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:55:25,941 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:25,941 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:25,941 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-10 04:55:25,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:25,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:55:25,941 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-10 04:55:25,941 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-10 04:55:25,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-10 04:55:25,941 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:25,941 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:25,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-10 04:55:25,942 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:25,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:55:25,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:25,942 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:25,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:25,942 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:25,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-10 04:55:25,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-10 04:55:25,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-10 04:55:25,942 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:25,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:55:25,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-10 04:55:25,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:25,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:25,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:25,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:55:25,944 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:25,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-10 04:55:25,944 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:25,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:25,944 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:25,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:25,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:25,946 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:25,946 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-10 04:55:25,946 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:25,946 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:25,946 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:25,947 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:25,947 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:25,947 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:25,947 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-05-10 04:55:25,947 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-10 04:55:25,947 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:55:25,947 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:25,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:25,946 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:25,946 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:25,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:25,947 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-10 04:55:25,947 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-05-10 04:55:25,947 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:25,947 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:25,947 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:25,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:25,947 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:25,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:25,947 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:25,947 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-10 04:55:25,947 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:25,948 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-10 04:55:25,948 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:25,948 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:25,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:25,948 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:25,948 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-10 04:55:25,948 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-10 04:55:26,048 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-10 04:55:26,049 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-10 04:55:26,059 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-10 04:55:26,060 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-05-10 04:55:26,060 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-05-10 04:55:26,060 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-05-10 04:55:26,060 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:55:26,060 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-05-10 04:55:26,060 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-05-10 04:55:26,060 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-10 04:55:26,060 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-10 04:55:26,062 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:26,062 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:26,062 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:26,062 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:26,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:26,062 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:26,062 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:26,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:26,062 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:26,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:26,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-10 04:55:26,062 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:26,062 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-10 04:55:26,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-10 04:55:26,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-10 04:55:26,062 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-10 04:55:26,063 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-10 04:55:26,063 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-10 04:55:26,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:55:26,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-10 04:55:26,063 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-05-10 04:55:26,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:55:26,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-10 04:55:26,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:55:26,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-10 04:55:26,063 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-05-10 04:55:26,063 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-05-10 04:55:26,063 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-05-10 04:55:26,063 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-05-10 04:55:26,063 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-05-10 04:55:26,063 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-05-10 04:55:26,063 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-05-10 04:55:26,063 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-05-10 04:55:26,064 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-05-10 04:55:26,064 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-05-10 04:55:26,064 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-05-10 04:55:26,064 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:55:26,063 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:55:26,064 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-05-10 04:55:26,064 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-05-10 04:55:26,064 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-05-10 04:55:26,064 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:55:26,064 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37995,1683694419499' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-05-10 04:55:26,064 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-05-10 04:55:26,064 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36665,1683694419452' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-05-10 04:55:26,064 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-05-10 04:55:26,064 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-10 04:55:26,064 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-10 04:55:26,064 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1683694525376.e9be1c226b4184298a60d5e1076cd600. 2023-05-10 04:55:26,064 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1683694525376.e9be1c226b4184298a60d5e1076cd600. started... 2023-05-10 04:55:26,064 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing e9be1c226b4184298a60d5e1076cd600 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-10 04:55:26,065 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-10 04:55:26,070 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-10 04:55:26,070 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-10 04:55:26,070 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:55:26,070 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-10 04:55:26,070 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:55:26,077 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/.tmp/cf1/34e676aeee2c4595a9dc599ec25aaf43 2023-05-10 04:55:26,091 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/.tmp/cf2/e29767cd1918445bbad6143229520f9e 2023-05-10 04:55:26,094 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/.tmp/cf1/34e676aeee2c4595a9dc599ec25aaf43 as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf1/34e676aeee2c4595a9dc599ec25aaf43 2023-05-10 04:55:26,098 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf1/34e676aeee2c4595a9dc599ec25aaf43, entries=10, sequenceid=19, filesize=5.0 K 2023-05-10 04:55:26,098 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/.tmp/cf2/e29767cd1918445bbad6143229520f9e as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf2/e29767cd1918445bbad6143229520f9e 2023-05-10 04:55:26,101 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf2/e29767cd1918445bbad6143229520f9e, entries=10, sequenceid=19, filesize=5.0 K 2023-05-10 04:55:26,102 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for e9be1c226b4184298a60d5e1076cd600 in 38ms, sequenceid=19, compaction requested=false 2023-05-10 04:55:26,102 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for e9be1c226b4184298a60d5e1076cd600: 2023-05-10 04:55:26,102 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1683694525376.e9be1c226b4184298a60d5e1076cd600. 2023-05-10 04:55:26,102 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-10 04:55:26,102 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-10 04:55:26,102 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:26,102 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-05-10 04:55:26,102 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37823,1683694419534' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-05-10 04:55:26,104 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-10 04:55:26,104 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:26,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:26,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:55:26,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:55:26,104 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-10 04:55:26,104 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:55:26,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:55:26,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:55:26,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-10 04:55:26,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:26,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:26,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:26,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:55:26,106 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37823,1683694419534' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-05-10 04:55:26,106 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@69ddfa17[Count = 0] remaining members to acquire global barrier 2023-05-10 04:55:26,106 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-05-10 04:55:26,106 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-10 04:55:26,107 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-10 04:55:26,107 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-10 04:55:26,107 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-10 04:55:26,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-10 04:55:26,107 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-10 04:55:26,107 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-05-10 04:55:26,107 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-10 04:55:26,107 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:26,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-10 04:55:26,107 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-10 04:55:26,107 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-05-10 04:55:26,107 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-05-10 04:55:26,107 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-10 04:55:26,107 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,36665,1683694419452' in zk 2023-05-10 04:55:26,107 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-05-10 04:55:26,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-10 04:55:26,107 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,37823,1683694419534' in zk 2023-05-10 04:55:26,108 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-05-10 04:55:26,108 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-05-10 04:55:26,108 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,37995,1683694419499' in zk 2023-05-10 04:55:26,109 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-05-10 04:55:26,109 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:55:26,109 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-05-10 04:55:26,110 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-05-10 04:55:26,110 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:55:26,110 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:26,110 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-05-10 04:55:26,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:26,111 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:55:26,110 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-05-10 04:55:26,111 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-05-10 04:55:26,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:55:26,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:55:26,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:55:26,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:55:26,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-10 04:55:26,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:26,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:26,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:26,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:55:26,112 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-10 04:55:26,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:26,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:26,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:26,113 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,37823,1683694419534': 2023-05-10 04:55:26,113 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37823,1683694419534' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-05-10 04:55:26,113 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-05-10 04:55:26,114 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-10 04:55:26,114 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-10 04:55:26,114 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-05-10 04:55:26,114 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-10 04:55:26,115 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-10 04:55:26,115 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-10 04:55:26,115 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-10 04:55:26,115 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-10 04:55:26,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-10 04:55:26,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-10 04:55:26,115 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:26,115 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-10 04:55:26,115 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:26,115 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-10 04:55:26,115 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:26,115 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-10 04:55:26,116 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:26,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-10 04:55:26,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:26,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-10 04:55:26,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:55:26,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:55:26,115 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:26,116 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:26,115 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:26,116 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:26,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-10 04:55:26,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-10 04:55:26,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:55:26,116 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:26,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:26,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-10 04:55:26,116 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:26,116 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-10 04:55:26,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:55:26,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-10 04:55:26,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:26,117 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:26,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:26,118 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:26,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:55:26,118 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:26,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-10 04:55:26,118 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:26,118 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:26,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:26,119 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:26,121 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:26,121 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:26,121 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:26,121 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-10 04:55:26,121 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:26,121 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:55:26,121 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:26,121 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:26,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:26,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:26,121 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-05-10 04:55:26,121 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-10 04:55:26,121 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:26,121 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:26,121 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:26,121 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:26,121 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:26,121 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-05-10 04:55:26,122 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-10 04:55:26,121 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:26,121 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:26,121 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:26,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:26,122 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-10 04:55:26,121 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:26,122 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:26,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:26,122 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-10 04:55:26,122 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:26,122 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:26,122 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-10 04:55:26,122 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-10 04:55:26,122 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:26,122 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:26,222 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-10 04:55:26,222 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-10 04:55:26,232 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-10 04:55:26,233 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-05-10 04:55:26,233 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-05-10 04:55:26,233 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-05-10 04:55:26,233 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:55:26,233 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-05-10 04:55:26,233 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-05-10 04:55:26,234 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-10 04:55:26,234 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-10 04:55:26,235 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:26,235 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:26,235 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:26,235 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:26,235 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:26,235 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:26,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:26,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:26,235 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:26,235 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:26,235 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-10 04:55:26,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:26,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-10 04:55:26,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-10 04:55:26,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-10 04:55:26,236 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-10 04:55:26,236 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-10 04:55:26,236 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-10 04:55:26,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:55:26,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-10 04:55:26,236 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-05-10 04:55:26,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:55:26,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-10 04:55:26,236 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-05-10 04:55:26,236 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-05-10 04:55:26,236 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-05-10 04:55:26,236 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-05-10 04:55:26,236 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:55:26,236 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-05-10 04:55:26,236 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-05-10 04:55:26,237 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-05-10 04:55:26,236 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-05-10 04:55:26,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:55:26,237 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-05-10 04:55:26,237 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36665,1683694419452' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-05-10 04:55:26,237 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:55:26,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-10 04:55:26,237 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-05-10 04:55:26,237 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-05-10 04:55:26,237 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-10 04:55:26,237 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-10 04:55:26,237 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1683694525376.e9be1c226b4184298a60d5e1076cd600. 2023-05-10 04:55:26,237 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1683694525376.e9be1c226b4184298a60d5e1076cd600. started... 2023-05-10 04:55:26,237 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing e9be1c226b4184298a60d5e1076cd600 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-10 04:55:26,238 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-05-10 04:55:26,238 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-05-10 04:55:26,238 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-05-10 04:55:26,238 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:55:26,238 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-05-10 04:55:26,238 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-05-10 04:55:26,238 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37995,1683694419499' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-05-10 04:55:26,240 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-10 04:55:26,243 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-10 04:55:26,243 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-10 04:55:26,244 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:55:26,244 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-10 04:55:26,244 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:55:26,250 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/.tmp/cf1/ec03059b17284629846b089825945254 2023-05-10 04:55:26,262 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/.tmp/cf2/dd83d4fecabb4f31af903d8b1e8934b8 2023-05-10 04:55:26,265 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/.tmp/cf1/ec03059b17284629846b089825945254 as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf1/ec03059b17284629846b089825945254 2023-05-10 04:55:26,269 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf1/ec03059b17284629846b089825945254, entries=10, sequenceid=32, filesize=5.0 K 2023-05-10 04:55:26,270 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/.tmp/cf2/dd83d4fecabb4f31af903d8b1e8934b8 as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf2/dd83d4fecabb4f31af903d8b1e8934b8 2023-05-10 04:55:26,274 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf2/dd83d4fecabb4f31af903d8b1e8934b8, entries=10, sequenceid=32, filesize=5.0 K 2023-05-10 04:55:26,275 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for e9be1c226b4184298a60d5e1076cd600 in 38ms, sequenceid=32, compaction requested=true 2023-05-10 04:55:26,275 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for e9be1c226b4184298a60d5e1076cd600: 2023-05-10 04:55:26,275 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1683694525376.e9be1c226b4184298a60d5e1076cd600. 2023-05-10 04:55:26,275 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-10 04:55:26,275 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-10 04:55:26,275 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:26,275 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-05-10 04:55:26,275 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37823,1683694419534' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-05-10 04:55:26,277 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-10 04:55:26,277 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:26,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:26,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:55:26,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:55:26,277 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-10 04:55:26,277 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:55:26,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:55:26,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:55:26,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-10 04:55:26,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:26,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:26,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:26,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:55:26,279 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37823,1683694419534' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-05-10 04:55:26,279 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@75114b7f[Count = 0] remaining members to acquire global barrier 2023-05-10 04:55:26,279 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-05-10 04:55:26,279 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-10 04:55:26,281 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-10 04:55:26,281 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-10 04:55:26,281 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-10 04:55:26,281 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-10 04:55:26,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-10 04:55:26,281 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-10 04:55:26,281 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-05-10 04:55:26,281 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-10 04:55:26,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-10 04:55:26,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-10 04:55:26,281 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-05-10 04:55:26,281 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-05-10 04:55:26,281 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-05-10 04:55:26,281 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-05-10 04:55:26,281 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:26,281 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-10 04:55:26,281 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-05-10 04:55:26,281 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,37823,1683694419534' in zk 2023-05-10 04:55:26,281 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,37995,1683694419499' in zk 2023-05-10 04:55:26,281 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,36665,1683694419452' in zk 2023-05-10 04:55:26,283 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-05-10 04:55:26,283 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:55:26,283 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-05-10 04:55:26,283 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:55:26,283 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-05-10 04:55:26,283 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-05-10 04:55:26,283 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-05-10 04:55:26,283 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:55:26,283 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:26,283 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-05-10 04:55:26,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:26,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:55:26,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:55:26,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:55:26,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:55:26,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-10 04:55:26,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:26,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:26,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:26,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:55:26,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-10 04:55:26,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:26,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:26,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:26,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,37823,1683694419534': 2023-05-10 04:55:26,286 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37823,1683694419534' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-05-10 04:55:26,286 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-05-10 04:55:26,287 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-10 04:55:26,287 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-10 04:55:26,287 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-05-10 04:55:26,287 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-10 04:55:26,288 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-10 04:55:26,288 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-10 04:55:26,288 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-10 04:55:26,288 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-10 04:55:26,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-10 04:55:26,288 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-10 04:55:26,288 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:26,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-10 04:55:26,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:55:26,288 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:26,288 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-10 04:55:26,288 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:26,288 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-10 04:55:26,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-10 04:55:26,288 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:26,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-10 04:55:26,288 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:26,289 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:26,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:55:26,289 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:26,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:26,289 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:26,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:26,289 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:26,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:55:26,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:26,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-10 04:55:26,289 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-10 04:55:26,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-10 04:55:26,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-10 04:55:26,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:55:26,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-10 04:55:26,290 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:26,291 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:26,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:26,291 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:26,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:26,291 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:26,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:55:26,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-10 04:55:26,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:26,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:26,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:26,295 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:26,295 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:26,295 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:26,295 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:26,295 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:26,295 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-10 04:55:26,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:26,295 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:26,295 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-05-10 04:55:26,295 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-10 04:55:26,295 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:26,295 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:26,295 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:26,295 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:26,295 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:55:26,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:26,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:26,295 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-05-10 04:55:26,295 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:26,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:26,295 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:26,295 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:26,296 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:26,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:26,295 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-10 04:55:26,296 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:26,296 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-10 04:55:26,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:26,296 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:26,296 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-10 04:55:26,296 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:26,296 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:26,296 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-10 04:55:26,296 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-10 04:55:26,396 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-10 04:55:26,397 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-10 04:55:26,406 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-10 04:55:26,408 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-05-10 04:55:26,408 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-05-10 04:55:26,408 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-05-10 04:55:26,408 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:55:26,408 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-05-10 04:55:26,408 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-05-10 04:55:26,408 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-10 04:55:26,408 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-10 04:55:26,409 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:26,409 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:26,409 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:26,409 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:26,409 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:26,410 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:26,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:26,410 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:26,410 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-10 04:55:26,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-10 04:55:26,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:26,409 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:26,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:26,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-10 04:55:26,410 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-10 04:55:26,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-10 04:55:26,411 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-10 04:55:26,411 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-10 04:55:26,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:55:26,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-10 04:55:26,411 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-05-10 04:55:26,411 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-05-10 04:55:26,411 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-05-10 04:55:26,411 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-05-10 04:55:26,411 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:55:26,411 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-05-10 04:55:26,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:55:26,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-10 04:55:26,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:55:26,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-10 04:55:26,411 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-05-10 04:55:26,412 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-05-10 04:55:26,412 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-05-10 04:55:26,412 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36665,1683694419452' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-05-10 04:55:26,412 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-05-10 04:55:26,412 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-05-10 04:55:26,412 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-05-10 04:55:26,412 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-05-10 04:55:26,412 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-05-10 04:55:26,412 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:55:26,412 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-05-10 04:55:26,412 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:55:26,412 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-05-10 04:55:26,412 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-05-10 04:55:26,412 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-05-10 04:55:26,412 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37995,1683694419499' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-05-10 04:55:26,412 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-10 04:55:26,412 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-10 04:55:26,412 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1683694525376.e9be1c226b4184298a60d5e1076cd600. 2023-05-10 04:55:26,413 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1683694525376.e9be1c226b4184298a60d5e1076cd600. started... 2023-05-10 04:55:26,413 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing e9be1c226b4184298a60d5e1076cd600 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-10 04:55:26,413 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-10 04:55:26,414 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-10 04:55:26,414 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:55:26,414 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-10 04:55:26,415 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-10 04:55:26,415 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:55:26,423 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/.tmp/cf1/91101e4c2e2e42578d390290caf782d5 2023-05-10 04:55:26,433 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/.tmp/cf2/57e73fe7c40143d6a3a6e5618f59ed0a 2023-05-10 04:55:26,436 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/.tmp/cf1/91101e4c2e2e42578d390290caf782d5 as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf1/91101e4c2e2e42578d390290caf782d5 2023-05-10 04:55:26,439 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf1/91101e4c2e2e42578d390290caf782d5, entries=10, sequenceid=45, filesize=5.0 K 2023-05-10 04:55:26,440 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/.tmp/cf2/57e73fe7c40143d6a3a6e5618f59ed0a as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf2/57e73fe7c40143d6a3a6e5618f59ed0a 2023-05-10 04:55:26,443 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf2/57e73fe7c40143d6a3a6e5618f59ed0a, entries=10, sequenceid=45, filesize=5.0 K 2023-05-10 04:55:26,444 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for e9be1c226b4184298a60d5e1076cd600 in 31ms, sequenceid=45, compaction requested=true 2023-05-10 04:55:26,444 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for e9be1c226b4184298a60d5e1076cd600: 2023-05-10 04:55:26,444 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1683694525376.e9be1c226b4184298a60d5e1076cd600. 2023-05-10 04:55:26,444 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-10 04:55:26,444 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-10 04:55:26,444 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:26,444 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-05-10 04:55:26,444 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37823,1683694419534' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-05-10 04:55:26,446 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-10 04:55:26,446 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:26,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:26,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:55:26,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:55:26,447 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-10 04:55:26,447 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:55:26,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:55:26,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:55:26,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-10 04:55:26,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:26,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:26,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:26,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:55:26,448 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37823,1683694419534' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-05-10 04:55:26,448 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@677ae35f[Count = 0] remaining members to acquire global barrier 2023-05-10 04:55:26,448 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-05-10 04:55:26,448 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-10 04:55:26,449 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-10 04:55:26,449 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-10 04:55:26,449 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-10 04:55:26,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-10 04:55:26,449 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-10 04:55:26,449 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-05-10 04:55:26,449 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-10 04:55:26,450 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-05-10 04:55:26,449 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:26,450 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-10 04:55:26,449 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-10 04:55:26,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-10 04:55:26,450 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,37995,1683694419499' in zk 2023-05-10 04:55:26,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-10 04:55:26,450 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-05-10 04:55:26,450 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-05-10 04:55:26,450 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-05-10 04:55:26,450 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,36665,1683694419452' in zk 2023-05-10 04:55:26,450 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-05-10 04:55:26,450 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,37823,1683694419534' in zk 2023-05-10 04:55:26,451 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-05-10 04:55:26,451 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:55:26,451 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-05-10 04:55:26,452 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-05-10 04:55:26,452 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:55:26,452 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:26,452 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-05-10 04:55:26,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:26,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:55:26,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:55:26,452 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-05-10 04:55:26,452 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:55:26,452 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-05-10 04:55:26,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:55:26,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:55:26,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-10 04:55:26,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:26,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:26,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:26,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:55:26,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-10 04:55:26,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:26,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:26,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:26,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,37823,1683694419534': 2023-05-10 04:55:26,455 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37823,1683694419534' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-05-10 04:55:26,455 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-05-10 04:55:26,455 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-10 04:55:26,455 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-10 04:55:26,455 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-05-10 04:55:26,455 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-10 04:55:26,458 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-10 04:55:26,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-10 04:55:26,458 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-10 04:55:26,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:55:26,458 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-10 04:55:26,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-10 04:55:26,458 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-10 04:55:26,458 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-10 04:55:26,458 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-10 04:55:26,458 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-10 04:55:26,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-10 04:55:26,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:55:26,458 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:26,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-10 04:55:26,458 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:26,458 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:26,458 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:26,458 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:26,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:26,458 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:26,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:26,458 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:26,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:26,459 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:26,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-10 04:55:26,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-10 04:55:26,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-10 04:55:26,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:55:26,467 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:26,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-10 04:55:26,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:55:26,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-10 04:55:26,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:26,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:26,468 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:26,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:26,469 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:26,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:55:26,469 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:26,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-10 04:55:26,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:26,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:26,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:26,474 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:26,474 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-10 04:55:26,474 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:26,474 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:26,474 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:26,474 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:26,474 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-05-10 04:55:26,474 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-10 04:55:26,474 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:26,474 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:26,474 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:55:26,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:26,474 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:26,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:26,474 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:26,475 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-05-10 04:55:26,475 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:26,475 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-10 04:55:26,475 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:26,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:26,475 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:26,475 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-10 04:55:26,475 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:26,475 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-10 04:55:26,475 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:26,475 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:26,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:26,475 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:26,475 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-10 04:55:26,475 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-10 04:55:26,476 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:26,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:26,479 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:26,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:26,575 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-10 04:55:26,576 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-10 04:55:26,586 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-10 04:55:26,587 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-05-10 04:55:26,587 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-05-10 04:55:26,587 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-05-10 04:55:26,587 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:55:26,587 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-05-10 04:55:26,587 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-05-10 04:55:26,588 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-10 04:55:26,588 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-10 04:55:26,589 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:26,589 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:26,589 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:26,589 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:26,589 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:26,589 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:26,589 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:26,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:26,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:26,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:26,590 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:26,590 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-10 04:55:26,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-10 04:55:26,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-10 04:55:26,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-10 04:55:26,590 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-10 04:55:26,590 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-10 04:55:26,590 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-10 04:55:26,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:55:26,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-10 04:55:26,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:55:26,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-10 04:55:26,590 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-05-10 04:55:26,590 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-05-10 04:55:26,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:55:26,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-10 04:55:26,591 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-05-10 04:55:26,591 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-05-10 04:55:26,591 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-05-10 04:55:26,591 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-05-10 04:55:26,591 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-05-10 04:55:26,591 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-05-10 04:55:26,591 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-05-10 04:55:26,591 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-05-10 04:55:26,591 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-05-10 04:55:26,591 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:55:26,591 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-05-10 04:55:26,591 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-05-10 04:55:26,591 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:55:26,591 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:55:26,591 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-05-10 04:55:26,591 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-05-10 04:55:26,591 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-05-10 04:55:26,591 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-05-10 04:55:26,591 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36665,1683694419452' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-05-10 04:55:26,591 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37995,1683694419499' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-05-10 04:55:26,591 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-10 04:55:26,591 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-10 04:55:26,591 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1683694525376.e9be1c226b4184298a60d5e1076cd600. 2023-05-10 04:55:26,591 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1683694525376.e9be1c226b4184298a60d5e1076cd600. started... 2023-05-10 04:55:26,591 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing e9be1c226b4184298a60d5e1076cd600 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-10 04:55:26,593 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-10 04:55:26,593 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-10 04:55:26,594 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-10 04:55:26,594 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:55:26,594 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-10 04:55:26,594 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:55:26,600 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/.tmp/cf1/af98ce1ca36546c289891c252022ca3f 2023-05-10 04:55:26,613 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/.tmp/cf2/2cc0f196bb124e2eaab06768c81709b6 2023-05-10 04:55:26,616 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/.tmp/cf1/af98ce1ca36546c289891c252022ca3f as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf1/af98ce1ca36546c289891c252022ca3f 2023-05-10 04:55:26,618 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf1/af98ce1ca36546c289891c252022ca3f, entries=10, sequenceid=58, filesize=5.0 K 2023-05-10 04:55:26,619 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/.tmp/cf2/2cc0f196bb124e2eaab06768c81709b6 as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf2/2cc0f196bb124e2eaab06768c81709b6 2023-05-10 04:55:26,621 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf2/2cc0f196bb124e2eaab06768c81709b6, entries=10, sequenceid=58, filesize=5.0 K 2023-05-10 04:55:26,622 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for e9be1c226b4184298a60d5e1076cd600 in 31ms, sequenceid=58, compaction requested=true 2023-05-10 04:55:26,622 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for e9be1c226b4184298a60d5e1076cd600: 2023-05-10 04:55:26,622 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1683694525376.e9be1c226b4184298a60d5e1076cd600. 2023-05-10 04:55:26,622 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-10 04:55:26,622 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-10 04:55:26,622 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:26,623 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-05-10 04:55:26,623 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37823,1683694419534' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-05-10 04:55:26,624 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-10 04:55:26,624 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:26,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:26,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:55:26,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:55:26,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:55:26,624 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-10 04:55:26,624 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:55:26,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:55:26,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-10 04:55:26,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:26,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:26,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:26,626 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:55:26,626 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37823,1683694419534' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-05-10 04:55:26,626 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@29efe8a1[Count = 0] remaining members to acquire global barrier 2023-05-10 04:55:26,626 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-05-10 04:55:26,626 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-10 04:55:26,627 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-10 04:55:26,627 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-10 04:55:26,627 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-10 04:55:26,627 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-10 04:55:26,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-10 04:55:26,627 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:26,627 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-10 04:55:26,627 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-10 04:55:26,627 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-05-10 04:55:26,627 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-10 04:55:26,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-10 04:55:26,627 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-10 04:55:26,627 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-05-10 04:55:26,627 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-05-10 04:55:26,628 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-05-10 04:55:26,628 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,37823,1683694419534' in zk 2023-05-10 04:55:26,627 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-05-10 04:55:26,628 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-05-10 04:55:26,628 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,37995,1683694419499' in zk 2023-05-10 04:55:26,628 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,36665,1683694419452' in zk 2023-05-10 04:55:26,629 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-05-10 04:55:26,629 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:26,629 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-05-10 04:55:26,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:26,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:55:26,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:55:26,629 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:55:26,629 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:55:26,629 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-05-10 04:55:26,629 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-05-10 04:55:26,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:55:26,630 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-05-10 04:55:26,630 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:55:26,631 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-05-10 04:55:26,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:55:26,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-10 04:55:26,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:26,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:26,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:26,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:55:26,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-10 04:55:26,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:26,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:26,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:26,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,37823,1683694419534': 2023-05-10 04:55:26,633 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37823,1683694419534' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-05-10 04:55:26,633 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-05-10 04:55:26,633 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-10 04:55:26,633 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-10 04:55:26,633 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-05-10 04:55:26,633 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-10 04:55:26,634 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-10 04:55:26,634 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-10 04:55:26,634 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:26,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-10 04:55:26,634 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-10 04:55:26,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:55:26,634 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-10 04:55:26,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-10 04:55:26,634 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-10 04:55:26,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-10 04:55:26,634 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-10 04:55:26,634 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:26,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:55:26,634 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:26,634 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-10 04:55:26,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-10 04:55:26,635 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:26,635 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:26,635 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:26,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:26,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:26,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:55:26,635 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:26,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:26,635 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:26,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-10 04:55:26,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-10 04:55:26,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-10 04:55:26,636 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:26,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:55:26,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-10 04:55:26,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:26,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:26,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:26,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:55:26,637 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:26,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-10 04:55:26,637 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:26,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:26,637 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:26,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:26,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:26,640 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:26,640 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-10 04:55:26,640 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:26,640 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:26,640 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:55:26,640 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:26,640 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:26,640 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-10 04:55:26,640 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:26,640 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:26,640 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-05-10 04:55:26,640 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-10 04:55:26,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:26,640 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:26,640 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:26,640 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:26,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:26,640 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-10 04:55:26,641 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:26,641 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:26,641 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-05-10 04:55:26,641 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:26,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:26,641 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-10 04:55:26,641 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-10 04:55:26,641 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:26,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:26,641 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-10 04:55:26,648 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:26,648 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:26,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:26,741 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-10 04:55:26,742 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-10 04:55:26,753 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(933): Client=jenkins//172.31.14.131 procedure request for: flush-table-proc 2023-05-10 04:55:26,754 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-05-10 04:55:26,754 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-05-10 04:55:26,754 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-05-10 04:55:26,754 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:55:26,754 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-05-10 04:55:26,754 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-05-10 04:55:26,755 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-10 04:55:26,755 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-10 04:55:26,756 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:26,756 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:26,756 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:26,756 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:26,756 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:26,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:26,756 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:26,756 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:26,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:26,756 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:26,756 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-05-10 04:55:26,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:26,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-10 04:55:26,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-10 04:55:26,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-10 04:55:26,757 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-10 04:55:26,757 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-10 04:55:26,757 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-10 04:55:26,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:55:26,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-10 04:55:26,757 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-05-10 04:55:26,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:55:26,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-10 04:55:26,758 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-05-10 04:55:26,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-05-10 04:55:26,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-10 04:55:26,758 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-05-10 04:55:26,759 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-05-10 04:55:26,759 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-05-10 04:55:26,759 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-05-10 04:55:26,759 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-05-10 04:55:26,759 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-05-10 04:55:26,759 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-05-10 04:55:26,759 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:55:26,759 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-05-10 04:55:26,759 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:55:26,759 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-05-10 04:55:26,759 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-05-10 04:55:26,759 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-05-10 04:55:26,760 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-05-10 04:55:26,760 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-05-10 04:55:26,760 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-05-10 04:55:26,760 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-05-10 04:55:26,760 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-05-10 04:55:26,760 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1683694525376.e9be1c226b4184298a60d5e1076cd600. 2023-05-10 04:55:26,760 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-05-10 04:55:26,760 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-05-10 04:55:26,760 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,36665,1683694419452' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-05-10 04:55:26,760 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1683694525376.e9be1c226b4184298a60d5e1076cd600. started... 2023-05-10 04:55:26,760 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37995,1683694419499' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-05-10 04:55:26,760 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing e9be1c226b4184298a60d5e1076cd600 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-05-10 04:55:26,762 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-10 04:55:26,763 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-10 04:55:26,763 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-10 04:55:26,763 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:55:26,763 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-10 04:55:26,763 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:55:26,776 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/.tmp/cf1/46158d7d3eea4447ab610c5263cb7a20 2023-05-10 04:55:26,789 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/.tmp/cf2/19631cfae8b94f0e9b5788c3a22fd03b 2023-05-10 04:55:26,793 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/.tmp/cf1/46158d7d3eea4447ab610c5263cb7a20 as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf1/46158d7d3eea4447ab610c5263cb7a20 2023-05-10 04:55:26,796 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf1/46158d7d3eea4447ab610c5263cb7a20, entries=10, sequenceid=71, filesize=5.0 K 2023-05-10 04:55:26,797 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/.tmp/cf2/19631cfae8b94f0e9b5788c3a22fd03b as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf2/19631cfae8b94f0e9b5788c3a22fd03b 2023-05-10 04:55:26,800 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf2/19631cfae8b94f0e9b5788c3a22fd03b, entries=10, sequenceid=71, filesize=5.0 K 2023-05-10 04:55:26,800 INFO [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for e9be1c226b4184298a60d5e1076cd600 in 40ms, sequenceid=71, compaction requested=true 2023-05-10 04:55:26,801 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for e9be1c226b4184298a60d5e1076cd600: 2023-05-10 04:55:26,801 DEBUG [rs(jenkins-hbase4.apache.org,37823,1683694419534)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1683694525376.e9be1c226b4184298a60d5e1076cd600. 2023-05-10 04:55:26,801 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-05-10 04:55:26,801 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-05-10 04:55:26,801 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:26,801 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-05-10 04:55:26,801 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase4.apache.org,37823,1683694419534' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-05-10 04:55:26,802 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:26,802 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-10 04:55:26,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:26,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:55:26,802 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:55:26,803 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-10 04:55:26,803 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-05-10 04:55:26,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:55:26,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:55:26,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-10 04:55:26,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:26,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:26,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:26,808 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:55:26,809 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase4.apache.org,37823,1683694419534' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-05-10 04:55:26,809 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@19bf86d8[Count = 0] remaining members to acquire global barrier 2023-05-10 04:55:26,809 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-05-10 04:55:26,809 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-10 04:55:26,810 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-10 04:55:26,810 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-10 04:55:26,810 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-10 04:55:26,810 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-10 04:55:26,810 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-10 04:55:26,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-10 04:55:26,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-10 04:55:26,810 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-05-10 04:55:26,810 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-05-10 04:55:26,810 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:26,810 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-05-10 04:55:26,810 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-05-10 04:55:26,810 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-05-10 04:55:26,810 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-10 04:55:26,810 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,36665,1683694419452' in zk 2023-05-10 04:55:26,810 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,37995,1683694419499' in zk 2023-05-10 04:55:26,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-10 04:55:26,811 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-05-10 04:55:26,811 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-05-10 04:55:26,811 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase4.apache.org,37823,1683694419534' in zk 2023-05-10 04:55:26,815 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-05-10 04:55:26,816 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:55:26,816 DEBUG [member: 'jenkins-hbase4.apache.org,37995,1683694419499' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-05-10 04:55:26,816 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-05-10 04:55:26,816 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:26,816 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-05-10 04:55:26,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:26,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:55:26,816 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:55:26,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:55:26,817 DEBUG [member: 'jenkins-hbase4.apache.org,36665,1683694419452' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-05-10 04:55:26,816 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:55:26,817 DEBUG [member: 'jenkins-hbase4.apache.org,37823,1683694419534' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-05-10 04:55:26,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:55:26,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:55:26,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-10 04:55:26,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:26,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:26,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:26,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:55:26,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-10 04:55:26,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:26,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:26,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:26,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase4.apache.org,37823,1683694419534': 2023-05-10 04:55:26,819 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase4.apache.org,37823,1683694419534' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-05-10 04:55:26,819 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-05-10 04:55:26,820 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-05-10 04:55:26,820 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-05-10 04:55:26,820 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-05-10 04:55:26,820 INFO [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-05-10 04:55:26,821 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-10 04:55:26,821 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-10 04:55:26,821 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-10 04:55:26,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-10 04:55:26,821 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-10 04:55:26,821 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-10 04:55:26,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-10 04:55:26,821 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:26,821 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-10 04:55:26,821 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:26,821 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-10 04:55:26,821 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:26,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-10 04:55:26,821 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:26,821 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:26,821 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-10 04:55:26,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-05-10 04:55:26,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-05-10 04:55:26,822 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:26,822 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:26,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:26,822 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:26,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:26,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-05-10 04:55:26,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:26,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-10 04:55:26,822 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:26,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-10 04:55:26,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-10 04:55:26,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-10 04:55:26,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-05-10 04:55:26,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-10 04:55:26,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:26,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:26,824 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:26,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:26,824 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:26,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-05-10 04:55:26,824 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:26,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-05-10 04:55:26,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:26,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:26,825 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:26,827 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:26,827 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:26,827 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:26,827 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:26,827 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:26,827 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-05-10 04:55:26,827 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:26,827 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-05-10 04:55:26,827 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-10 04:55:26,827 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-05-10 04:55:26,827 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:26,827 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-05-10 04:55:26,827 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-05-10 04:55:26,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:26,827 DEBUG [(jenkins-hbase4.apache.org,37903,1683694418326)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-05-10 04:55:26,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:26,828 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-05-10 04:55:26,828 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-05-10 04:55:26,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-05-10 04:55:26,827 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:26,828 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:26,828 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-05-10 04:55:26,828 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:26,828 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-10 04:55:26,828 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:26,828 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:26,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:26,828 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:26,828 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-05-10 04:55:26,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:26,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-05-10 04:55:26,828 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:26,828 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-05-10 04:55:26,828 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-05-10 04:55:26,928 DEBUG [Listener at localhost/45007] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-05-10 04:55:26,929 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(1186): Checking to see if procedure from request:flush-table-proc is done 2023-05-10 04:55:26,935 DEBUG [Listener at localhost/45007] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:26,936 DEBUG [Listener at localhost/45007] procedure2.ProcedureExecutor(1029): Stored pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=e9be1c226b4184298a60d5e1076cd600, daughterA=48970235bef33aa5251ac68151d7d8b4, daughterB=c5e0f2bc1f34b2b275d44dfbd485b943 2023-05-10 04:55:26,937 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=e9be1c226b4184298a60d5e1076cd600, daughterA=48970235bef33aa5251ac68151d7d8b4, daughterB=c5e0f2bc1f34b2b275d44dfbd485b943 2023-05-10 04:55:26,937 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=e9be1c226b4184298a60d5e1076cd600, daughterA=48970235bef33aa5251ac68151d7d8b4, daughterB=c5e0f2bc1f34b2b275d44dfbd485b943 2023-05-10 04:55:26,937 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=e9be1c226b4184298a60d5e1076cd600, daughterA=48970235bef33aa5251ac68151d7d8b4, daughterB=c5e0f2bc1f34b2b275d44dfbd485b943 2023-05-10 04:55:26,941 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=145, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=e9be1c226b4184298a60d5e1076cd600, UNASSIGN}] 2023-05-10 04:55:26,941 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=145, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=e9be1c226b4184298a60d5e1076cd600, UNASSIGN 2023-05-10 04:55:26,942 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=145 updating hbase:meta row=e9be1c226b4184298a60d5e1076cd600, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:26,942 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1683694525376.e9be1c226b4184298a60d5e1076cd600.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1683694526942"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683694526942"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683694526942"}]},"ts":"1683694526942"} 2023-05-10 04:55:26,943 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=146, ppid=145, state=RUNNABLE; CloseRegionProcedure e9be1c226b4184298a60d5e1076cd600, server=jenkins-hbase4.apache.org,37823,1683694419534}] 2023-05-10 04:55:27,095 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close e9be1c226b4184298a60d5e1076cd600 2023-05-10 04:55:27,095 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing e9be1c226b4184298a60d5e1076cd600, disabling compactions & flushes 2023-05-10 04:55:27,095 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegion,,1683694525376.e9be1c226b4184298a60d5e1076cd600. 2023-05-10 04:55:27,095 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegion,,1683694525376.e9be1c226b4184298a60d5e1076cd600. 2023-05-10 04:55:27,095 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegion,,1683694525376.e9be1c226b4184298a60d5e1076cd600. after waiting 0 ms 2023-05-10 04:55:27,095 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegion,,1683694525376.e9be1c226b4184298a60d5e1076cd600. 2023-05-10 04:55:27,095 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing e9be1c226b4184298a60d5e1076cd600 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-05-10 04:55:27,104 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/.tmp/cf1/e7a24003dc044b82b9d96bae6d63b040 2023-05-10 04:55:27,115 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/.tmp/cf2/30a77dd573b64ac598ec88e3afa961e4 2023-05-10 04:55:27,119 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/.tmp/cf1/e7a24003dc044b82b9d96bae6d63b040 as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf1/e7a24003dc044b82b9d96bae6d63b040 2023-05-10 04:55:27,122 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf1/e7a24003dc044b82b9d96bae6d63b040, entries=8, sequenceid=82, filesize=5.0 K 2023-05-10 04:55:27,122 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/.tmp/cf2/30a77dd573b64ac598ec88e3afa961e4 as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf2/30a77dd573b64ac598ec88e3afa961e4 2023-05-10 04:55:27,125 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf2/30a77dd573b64ac598ec88e3afa961e4, entries=8, sequenceid=82, filesize=5.0 K 2023-05-10 04:55:27,126 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for e9be1c226b4184298a60d5e1076cd600 in 31ms, sequenceid=82, compaction requested=true 2023-05-10 04:55:27,130 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-05-10 04:55:27,131 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-10 04:55:27,131 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegion,,1683694525376.e9be1c226b4184298a60d5e1076cd600. 2023-05-10 04:55:27,131 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for e9be1c226b4184298a60d5e1076cd600: 2023-05-10 04:55:27,132 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed e9be1c226b4184298a60d5e1076cd600 2023-05-10 04:55:27,133 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=145 updating hbase:meta row=e9be1c226b4184298a60d5e1076cd600, regionState=CLOSED 2023-05-10 04:55:27,133 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegion,,1683694525376.e9be1c226b4184298a60d5e1076cd600.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1683694527132"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683694527132"}]},"ts":"1683694527132"} 2023-05-10 04:55:27,135 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=146, resume processing ppid=145 2023-05-10 04:55:27,135 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=146, ppid=145, state=SUCCESS; CloseRegionProcedure e9be1c226b4184298a60d5e1076cd600, server=jenkins-hbase4.apache.org,37823,1683694419534 in 191 msec 2023-05-10 04:55:27,136 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=145, resume processing ppid=144 2023-05-10 04:55:27,136 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=145, ppid=144, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=e9be1c226b4184298a60d5e1076cd600, UNASSIGN in 194 msec 2023-05-10 04:55:27,145 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=144 splitting 14 storefiles, region=e9be1c226b4184298a60d5e1076cd600, threads=14 2023-05-10 04:55:27,146 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf2/19631cfae8b94f0e9b5788c3a22fd03b for region: e9be1c226b4184298a60d5e1076cd600 2023-05-10 04:55:27,146 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf2/2cc0f196bb124e2eaab06768c81709b6 for region: e9be1c226b4184298a60d5e1076cd600 2023-05-10 04:55:27,146 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf2/30a77dd573b64ac598ec88e3afa961e4 for region: e9be1c226b4184298a60d5e1076cd600 2023-05-10 04:55:27,146 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf2/4664a731a30c46e2bf2e8bad410a89bd for region: e9be1c226b4184298a60d5e1076cd600 2023-05-10 04:55:27,146 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf2/57e73fe7c40143d6a3a6e5618f59ed0a for region: e9be1c226b4184298a60d5e1076cd600 2023-05-10 04:55:27,146 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf2/dd83d4fecabb4f31af903d8b1e8934b8 for region: e9be1c226b4184298a60d5e1076cd600 2023-05-10 04:55:27,146 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf2/e29767cd1918445bbad6143229520f9e for region: e9be1c226b4184298a60d5e1076cd600 2023-05-10 04:55:27,146 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf1/34e676aeee2c4595a9dc599ec25aaf43 for region: e9be1c226b4184298a60d5e1076cd600 2023-05-10 04:55:27,147 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf1/46158d7d3eea4447ab610c5263cb7a20 for region: e9be1c226b4184298a60d5e1076cd600 2023-05-10 04:55:27,151 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf1/58d433c9ed254876ac1bfa1b33856169 for region: e9be1c226b4184298a60d5e1076cd600 2023-05-10 04:55:27,151 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf1/91101e4c2e2e42578d390290caf782d5 for region: e9be1c226b4184298a60d5e1076cd600 2023-05-10 04:55:27,151 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf1/af98ce1ca36546c289891c252022ca3f for region: e9be1c226b4184298a60d5e1076cd600 2023-05-10 04:55:27,151 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf1/e7a24003dc044b82b9d96bae6d63b040 for region: e9be1c226b4184298a60d5e1076cd600 2023-05-10 04:55:27,151 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf1/ec03059b17284629846b089825945254 for region: e9be1c226b4184298a60d5e1076cd600 2023-05-10 04:55:27,303 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf1/e7a24003dc044b82b9d96bae6d63b040 for region: e9be1c226b4184298a60d5e1076cd600 2023-05-10 04:55:27,315 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf1/34e676aeee2c4595a9dc599ec25aaf43 for region: e9be1c226b4184298a60d5e1076cd600 2023-05-10 04:55:27,316 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf2/4664a731a30c46e2bf2e8bad410a89bd for region: e9be1c226b4184298a60d5e1076cd600 2023-05-10 04:55:27,370 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf2/dd83d4fecabb4f31af903d8b1e8934b8 for region: e9be1c226b4184298a60d5e1076cd600 2023-05-10 04:55:27,383 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf2/e29767cd1918445bbad6143229520f9e for region: e9be1c226b4184298a60d5e1076cd600 2023-05-10 04:55:27,395 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf1/91101e4c2e2e42578d390290caf782d5 for region: e9be1c226b4184298a60d5e1076cd600 2023-05-10 04:55:27,396 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf1/af98ce1ca36546c289891c252022ca3f for region: e9be1c226b4184298a60d5e1076cd600 2023-05-10 04:55:27,396 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf1/ec03059b17284629846b089825945254 for region: e9be1c226b4184298a60d5e1076cd600 2023-05-10 04:55:27,398 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf2/57e73fe7c40143d6a3a6e5618f59ed0a for region: e9be1c226b4184298a60d5e1076cd600 2023-05-10 04:55:27,398 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf1/46158d7d3eea4447ab610c5263cb7a20 for region: e9be1c226b4184298a60d5e1076cd600 2023-05-10 04:55:27,399 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf2/30a77dd573b64ac598ec88e3afa961e4 for region: e9be1c226b4184298a60d5e1076cd600 2023-05-10 04:55:27,399 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf2/19631cfae8b94f0e9b5788c3a22fd03b for region: e9be1c226b4184298a60d5e1076cd600 2023-05-10 04:55:27,400 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf1/58d433c9ed254876ac1bfa1b33856169 for region: e9be1c226b4184298a60d5e1076cd600 2023-05-10 04:55:27,400 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf2/2cc0f196bb124e2eaab06768c81709b6 for region: e9be1c226b4184298a60d5e1076cd600 2023-05-10 04:55:27,400 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=144 split storefiles for region e9be1c226b4184298a60d5e1076cd600 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-05-10 04:55:27,435 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/48970235bef33aa5251ac68151d7d8b4/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-05-10 04:55:27,436 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/c5e0f2bc1f34b2b275d44dfbd485b943/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-05-10 04:55:27,438 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegion,,1683694525376.e9be1c226b4184298a60d5e1076cd600.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1683694527438"},{"qualifier":"splitA","vlen":56,"tag":[],"timestamp":"1683694527438"},{"qualifier":"splitB","vlen":56,"tag":[],"timestamp":"1683694527438"}]},"ts":"1683694527438"} 2023-05-10 04:55:27,438 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegion,,1683694526936.48970235bef33aa5251ac68151d7d8b4.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1683694527438"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683694527438"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683694527438"}]},"ts":"1683694527438"} 2023-05-10 04:55:27,438 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegion,41,1683694526936.c5e0f2bc1f34b2b275d44dfbd485b943.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1683694527438"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683694527438"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683694527438"}]},"ts":"1683694527438"} 2023-05-10 04:55:27,452 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=147, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=48970235bef33aa5251ac68151d7d8b4, ASSIGN}, {pid=148, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=c5e0f2bc1f34b2b275d44dfbd485b943, ASSIGN}] 2023-05-10 04:55:27,453 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=148, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=c5e0f2bc1f34b2b275d44dfbd485b943, ASSIGN 2023-05-10 04:55:27,453 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=148, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegion, region=c5e0f2bc1f34b2b275d44dfbd485b943, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,37823,1683694419534; forceNewPlan=false, retain=false 2023-05-10 04:55:27,454 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=147, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=48970235bef33aa5251ac68151d7d8b4, ASSIGN 2023-05-10 04:55:27,454 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=147, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegion, region=48970235bef33aa5251ac68151d7d8b4, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase4.apache.org,37823,1683694419534; forceNewPlan=false, retain=false 2023-05-10 04:55:27,603 INFO [jenkins-hbase4:37903] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-05-10 04:55:27,605 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=147 updating hbase:meta row=48970235bef33aa5251ac68151d7d8b4, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:27,606 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1683694526936.48970235bef33aa5251ac68151d7d8b4.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1683694527605"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683694527605"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683694527605"}]},"ts":"1683694527605"} 2023-05-10 04:55:27,607 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=149, ppid=147, state=RUNNABLE; OpenRegionProcedure 48970235bef33aa5251ac68151d7d8b4, server=jenkins-hbase4.apache.org,37823,1683694419534}] 2023-05-10 04:55:27,609 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=148 updating hbase:meta row=c5e0f2bc1f34b2b275d44dfbd485b943, regionState=OPENING, regionLocation=jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:27,609 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,41,1683694526936.c5e0f2bc1f34b2b275d44dfbd485b943.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1683694527609"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683694527609"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683694527609"}]},"ts":"1683694527609"} 2023-05-10 04:55:27,610 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=150, ppid=148, state=RUNNABLE; OpenRegionProcedure c5e0f2bc1f34b2b275d44dfbd485b943, server=jenkins-hbase4.apache.org,37823,1683694419534}] 2023-05-10 04:55:27,761 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegion,,1683694526936.48970235bef33aa5251ac68151d7d8b4. 2023-05-10 04:55:27,761 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 48970235bef33aa5251ac68151d7d8b4, NAME => 'testSplitTableRegion,,1683694526936.48970235bef33aa5251ac68151d7d8b4.', STARTKEY => '', ENDKEY => '41'} 2023-05-10 04:55:27,761 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-05-10 04:55:27,762 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegion 48970235bef33aa5251ac68151d7d8b4 2023-05-10 04:55:27,762 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegion,,1683694526936.48970235bef33aa5251ac68151d7d8b4.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-10 04:55:27,762 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for 48970235bef33aa5251ac68151d7d8b4 2023-05-10 04:55:27,762 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for 48970235bef33aa5251ac68151d7d8b4 2023-05-10 04:55:27,763 INFO [StoreOpener-48970235bef33aa5251ac68151d7d8b4-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 48970235bef33aa5251ac68151d7d8b4 2023-05-10 04:55:27,764 DEBUG [StoreOpener-48970235bef33aa5251ac68151d7d8b4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/48970235bef33aa5251ac68151d7d8b4/cf1 2023-05-10 04:55:27,764 DEBUG [StoreOpener-48970235bef33aa5251ac68151d7d8b4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/48970235bef33aa5251ac68151d7d8b4/cf1 2023-05-10 04:55:27,764 INFO [StoreOpener-48970235bef33aa5251ac68151d7d8b4-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 48970235bef33aa5251ac68151d7d8b4 columnFamilyName cf1 2023-05-10 04:55:27,775 DEBUG [StoreOpener-48970235bef33aa5251ac68151d7d8b4-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/48970235bef33aa5251ac68151d7d8b4/cf1/34e676aeee2c4595a9dc599ec25aaf43.e9be1c226b4184298a60d5e1076cd600->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf1/34e676aeee2c4595a9dc599ec25aaf43-bottom 2023-05-10 04:55:27,779 DEBUG [StoreOpener-48970235bef33aa5251ac68151d7d8b4-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/48970235bef33aa5251ac68151d7d8b4/cf1/46158d7d3eea4447ab610c5263cb7a20.e9be1c226b4184298a60d5e1076cd600->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf1/46158d7d3eea4447ab610c5263cb7a20-bottom 2023-05-10 04:55:27,783 DEBUG [StoreOpener-48970235bef33aa5251ac68151d7d8b4-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/48970235bef33aa5251ac68151d7d8b4/cf1/58d433c9ed254876ac1bfa1b33856169.e9be1c226b4184298a60d5e1076cd600->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf1/58d433c9ed254876ac1bfa1b33856169-bottom 2023-05-10 04:55:27,786 DEBUG [StoreOpener-48970235bef33aa5251ac68151d7d8b4-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/48970235bef33aa5251ac68151d7d8b4/cf1/91101e4c2e2e42578d390290caf782d5.e9be1c226b4184298a60d5e1076cd600->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf1/91101e4c2e2e42578d390290caf782d5-bottom 2023-05-10 04:55:27,789 DEBUG [StoreOpener-48970235bef33aa5251ac68151d7d8b4-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/48970235bef33aa5251ac68151d7d8b4/cf1/af98ce1ca36546c289891c252022ca3f.e9be1c226b4184298a60d5e1076cd600->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf1/af98ce1ca36546c289891c252022ca3f-bottom 2023-05-10 04:55:27,793 DEBUG [StoreOpener-48970235bef33aa5251ac68151d7d8b4-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/48970235bef33aa5251ac68151d7d8b4/cf1/e7a24003dc044b82b9d96bae6d63b040.e9be1c226b4184298a60d5e1076cd600->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf1/e7a24003dc044b82b9d96bae6d63b040-bottom 2023-05-10 04:55:27,796 DEBUG [StoreOpener-48970235bef33aa5251ac68151d7d8b4-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/48970235bef33aa5251ac68151d7d8b4/cf1/ec03059b17284629846b089825945254.e9be1c226b4184298a60d5e1076cd600->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf1/ec03059b17284629846b089825945254-bottom 2023-05-10 04:55:27,796 INFO [StoreOpener-48970235bef33aa5251ac68151d7d8b4-1] regionserver.HStore(310): Store=48970235bef33aa5251ac68151d7d8b4/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-10 04:55:27,796 INFO [StoreOpener-48970235bef33aa5251ac68151d7d8b4-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 48970235bef33aa5251ac68151d7d8b4 2023-05-10 04:55:27,797 DEBUG [StoreOpener-48970235bef33aa5251ac68151d7d8b4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/48970235bef33aa5251ac68151d7d8b4/cf2 2023-05-10 04:55:27,797 DEBUG [StoreOpener-48970235bef33aa5251ac68151d7d8b4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/48970235bef33aa5251ac68151d7d8b4/cf2 2023-05-10 04:55:27,797 INFO [StoreOpener-48970235bef33aa5251ac68151d7d8b4-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 48970235bef33aa5251ac68151d7d8b4 columnFamilyName cf2 2023-05-10 04:55:27,810 DEBUG [StoreOpener-48970235bef33aa5251ac68151d7d8b4-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/48970235bef33aa5251ac68151d7d8b4/cf2/19631cfae8b94f0e9b5788c3a22fd03b.e9be1c226b4184298a60d5e1076cd600->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf2/19631cfae8b94f0e9b5788c3a22fd03b-bottom 2023-05-10 04:55:27,814 DEBUG [StoreOpener-48970235bef33aa5251ac68151d7d8b4-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/48970235bef33aa5251ac68151d7d8b4/cf2/2cc0f196bb124e2eaab06768c81709b6.e9be1c226b4184298a60d5e1076cd600->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf2/2cc0f196bb124e2eaab06768c81709b6-bottom 2023-05-10 04:55:27,818 DEBUG [StoreOpener-48970235bef33aa5251ac68151d7d8b4-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/48970235bef33aa5251ac68151d7d8b4/cf2/30a77dd573b64ac598ec88e3afa961e4.e9be1c226b4184298a60d5e1076cd600->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf2/30a77dd573b64ac598ec88e3afa961e4-bottom 2023-05-10 04:55:27,821 DEBUG [StoreOpener-48970235bef33aa5251ac68151d7d8b4-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/48970235bef33aa5251ac68151d7d8b4/cf2/4664a731a30c46e2bf2e8bad410a89bd.e9be1c226b4184298a60d5e1076cd600->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf2/4664a731a30c46e2bf2e8bad410a89bd-bottom 2023-05-10 04:55:27,825 DEBUG [StoreOpener-48970235bef33aa5251ac68151d7d8b4-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/48970235bef33aa5251ac68151d7d8b4/cf2/57e73fe7c40143d6a3a6e5618f59ed0a.e9be1c226b4184298a60d5e1076cd600->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf2/57e73fe7c40143d6a3a6e5618f59ed0a-bottom 2023-05-10 04:55:27,828 DEBUG [StoreOpener-48970235bef33aa5251ac68151d7d8b4-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/48970235bef33aa5251ac68151d7d8b4/cf2/dd83d4fecabb4f31af903d8b1e8934b8.e9be1c226b4184298a60d5e1076cd600->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf2/dd83d4fecabb4f31af903d8b1e8934b8-bottom 2023-05-10 04:55:27,832 DEBUG [StoreOpener-48970235bef33aa5251ac68151d7d8b4-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/48970235bef33aa5251ac68151d7d8b4/cf2/e29767cd1918445bbad6143229520f9e.e9be1c226b4184298a60d5e1076cd600->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf2/e29767cd1918445bbad6143229520f9e-bottom 2023-05-10 04:55:27,832 INFO [StoreOpener-48970235bef33aa5251ac68151d7d8b4-1] regionserver.HStore(310): Store=48970235bef33aa5251ac68151d7d8b4/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-10 04:55:27,833 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/48970235bef33aa5251ac68151d7d8b4 2023-05-10 04:55:27,834 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/48970235bef33aa5251ac68151d7d8b4 2023-05-10 04:55:27,835 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegion descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-05-10 04:55:27,836 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for 48970235bef33aa5251ac68151d7d8b4 2023-05-10 04:55:27,837 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened 48970235bef33aa5251ac68151d7d8b4; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=68339996, jitterRate=0.018345296382904053}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-05-10 04:55:27,837 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for 48970235bef33aa5251ac68151d7d8b4: 2023-05-10 04:55:27,838 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2334): Post open deploy tasks for testSplitTableRegion,,1683694526936.48970235bef33aa5251ac68151d7d8b4., pid=149, masterSystemTime=1683694527758 2023-05-10 04:55:27,838 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-10 04:55:27,839 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-10 04:55:27,839 DEBUG [RS:2;jenkins-hbase4:37823-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-05-10 04:55:27,839 DEBUG [RS:2;jenkins-hbase4:37823-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-05-10 04:55:27,839 INFO [RS:2;jenkins-hbase4:37823-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegion,,1683694526936.48970235bef33aa5251ac68151d7d8b4. 2023-05-10 04:55:27,839 DEBUG [RS:2;jenkins-hbase4:37823-shortCompactions-0] regionserver.HStore(1912): 48970235bef33aa5251ac68151d7d8b4/cf1 is initiating minor compaction (all files) 2023-05-10 04:55:27,839 INFO [RS:2;jenkins-hbase4:37823-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegion,,1683694526936.48970235bef33aa5251ac68151d7d8b4. 2023-05-10 04:55:27,839 DEBUG [RS:2;jenkins-hbase4:37823-longCompactions-0] regionserver.HStore(1912): 48970235bef33aa5251ac68151d7d8b4/cf2 is initiating minor compaction (all files) 2023-05-10 04:55:27,839 INFO [RS:2;jenkins-hbase4:37823-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 48970235bef33aa5251ac68151d7d8b4/cf1 in testSplitTableRegion,,1683694526936.48970235bef33aa5251ac68151d7d8b4. 2023-05-10 04:55:27,839 INFO [RS:2;jenkins-hbase4:37823-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 48970235bef33aa5251ac68151d7d8b4/cf2 in testSplitTableRegion,,1683694526936.48970235bef33aa5251ac68151d7d8b4. 2023-05-10 04:55:27,839 INFO [RS:2;jenkins-hbase4:37823-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/48970235bef33aa5251ac68151d7d8b4/cf2/4664a731a30c46e2bf2e8bad410a89bd.e9be1c226b4184298a60d5e1076cd600->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf2/4664a731a30c46e2bf2e8bad410a89bd-bottom, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/48970235bef33aa5251ac68151d7d8b4/cf2/e29767cd1918445bbad6143229520f9e.e9be1c226b4184298a60d5e1076cd600->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf2/e29767cd1918445bbad6143229520f9e-bottom, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/48970235bef33aa5251ac68151d7d8b4/cf2/dd83d4fecabb4f31af903d8b1e8934b8.e9be1c226b4184298a60d5e1076cd600->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf2/dd83d4fecabb4f31af903d8b1e8934b8-bottom, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/48970235bef33aa5251ac68151d7d8b4/cf2/57e73fe7c40143d6a3a6e5618f59ed0a.e9be1c226b4184298a60d5e1076cd600->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf2/57e73fe7c40143d6a3a6e5618f59ed0a-bottom, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/48970235bef33aa5251ac68151d7d8b4/cf2/2cc0f196bb124e2eaab06768c81709b6.e9be1c226b4184298a60d5e1076cd600->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf2/2cc0f196bb124e2eaab06768c81709b6-bottom, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/48970235bef33aa5251ac68151d7d8b4/cf2/19631cfae8b94f0e9b5788c3a22fd03b.e9be1c226b4184298a60d5e1076cd600->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf2/19631cfae8b94f0e9b5788c3a22fd03b-bottom, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/48970235bef33aa5251ac68151d7d8b4/cf2/30a77dd573b64ac598ec88e3afa961e4.e9be1c226b4184298a60d5e1076cd600->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf2/30a77dd573b64ac598ec88e3afa961e4-bottom] into tmpdir=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/48970235bef33aa5251ac68151d7d8b4/.tmp, totalSize=34.9 K 2023-05-10 04:55:27,840 INFO [RS:2;jenkins-hbase4:37823-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/48970235bef33aa5251ac68151d7d8b4/cf1/58d433c9ed254876ac1bfa1b33856169.e9be1c226b4184298a60d5e1076cd600->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf1/58d433c9ed254876ac1bfa1b33856169-bottom, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/48970235bef33aa5251ac68151d7d8b4/cf1/34e676aeee2c4595a9dc599ec25aaf43.e9be1c226b4184298a60d5e1076cd600->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf1/34e676aeee2c4595a9dc599ec25aaf43-bottom, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/48970235bef33aa5251ac68151d7d8b4/cf1/ec03059b17284629846b089825945254.e9be1c226b4184298a60d5e1076cd600->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf1/ec03059b17284629846b089825945254-bottom, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/48970235bef33aa5251ac68151d7d8b4/cf1/91101e4c2e2e42578d390290caf782d5.e9be1c226b4184298a60d5e1076cd600->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf1/91101e4c2e2e42578d390290caf782d5-bottom, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/48970235bef33aa5251ac68151d7d8b4/cf1/af98ce1ca36546c289891c252022ca3f.e9be1c226b4184298a60d5e1076cd600->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf1/af98ce1ca36546c289891c252022ca3f-bottom, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/48970235bef33aa5251ac68151d7d8b4/cf1/46158d7d3eea4447ab610c5263cb7a20.e9be1c226b4184298a60d5e1076cd600->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf1/46158d7d3eea4447ab610c5263cb7a20-bottom, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/48970235bef33aa5251ac68151d7d8b4/cf1/e7a24003dc044b82b9d96bae6d63b040.e9be1c226b4184298a60d5e1076cd600->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf1/e7a24003dc044b82b9d96bae6d63b040-bottom] into tmpdir=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/48970235bef33aa5251ac68151d7d8b4/.tmp, totalSize=34.9 K 2023-05-10 04:55:27,840 DEBUG [RS:2;jenkins-hbase4:37823-longCompactions-0] compactions.Compactor(207): Compacting 4664a731a30c46e2bf2e8bad410a89bd.e9be1c226b4184298a60d5e1076cd600, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1683694525880 2023-05-10 04:55:27,840 DEBUG [RS:2;jenkins-hbase4:37823-shortCompactions-0] compactions.Compactor(207): Compacting 58d433c9ed254876ac1bfa1b33856169.e9be1c226b4184298a60d5e1076cd600, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1683694525880 2023-05-10 04:55:27,840 DEBUG [RS:2;jenkins-hbase4:37823-longCompactions-0] compactions.Compactor(207): Compacting e29767cd1918445bbad6143229520f9e.e9be1c226b4184298a60d5e1076cd600, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1683694526050 2023-05-10 04:55:27,843 DEBUG [RS:2;jenkins-hbase4:37823-longCompactions-0] compactions.Compactor(207): Compacting dd83d4fecabb4f31af903d8b1e8934b8.e9be1c226b4184298a60d5e1076cd600, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1683694526223 2023-05-10 04:55:27,843 DEBUG [RS:2;jenkins-hbase4:37823-shortCompactions-0] compactions.Compactor(207): Compacting 34e676aeee2c4595a9dc599ec25aaf43.e9be1c226b4184298a60d5e1076cd600, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1683694526050 2023-05-10 04:55:27,843 DEBUG [RS:2;jenkins-hbase4:37823-longCompactions-0] compactions.Compactor(207): Compacting 57e73fe7c40143d6a3a6e5618f59ed0a.e9be1c226b4184298a60d5e1076cd600, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1683694526398 2023-05-10 04:55:27,843 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2361): Finished post open deploy task for testSplitTableRegion,,1683694526936.48970235bef33aa5251ac68151d7d8b4. 2023-05-10 04:55:27,843 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegion,,1683694526936.48970235bef33aa5251ac68151d7d8b4. 2023-05-10 04:55:27,843 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegion,41,1683694526936.c5e0f2bc1f34b2b275d44dfbd485b943. 2023-05-10 04:55:27,843 DEBUG [RS:2;jenkins-hbase4:37823-shortCompactions-0] compactions.Compactor(207): Compacting ec03059b17284629846b089825945254.e9be1c226b4184298a60d5e1076cd600, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1683694526223 2023-05-10 04:55:27,843 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => c5e0f2bc1f34b2b275d44dfbd485b943, NAME => 'testSplitTableRegion,41,1683694526936.c5e0f2bc1f34b2b275d44dfbd485b943.', STARTKEY => '41', ENDKEY => ''} 2023-05-10 04:55:27,844 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-05-10 04:55:27,844 DEBUG [RS:2;jenkins-hbase4:37823-shortCompactions-0] compactions.Compactor(207): Compacting 91101e4c2e2e42578d390290caf782d5.e9be1c226b4184298a60d5e1076cd600, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1683694526398 2023-05-10 04:55:27,844 DEBUG [RS:2;jenkins-hbase4:37823-longCompactions-0] compactions.Compactor(207): Compacting 2cc0f196bb124e2eaab06768c81709b6.e9be1c226b4184298a60d5e1076cd600, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1683694526577 2023-05-10 04:55:27,844 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegion c5e0f2bc1f34b2b275d44dfbd485b943 2023-05-10 04:55:27,844 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegion,41,1683694526936.c5e0f2bc1f34b2b275d44dfbd485b943.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-05-10 04:55:27,844 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7894): checking encryption for c5e0f2bc1f34b2b275d44dfbd485b943 2023-05-10 04:55:27,844 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(7897): checking classloading for c5e0f2bc1f34b2b275d44dfbd485b943 2023-05-10 04:55:27,844 DEBUG [RS:2;jenkins-hbase4:37823-shortCompactions-0] compactions.Compactor(207): Compacting af98ce1ca36546c289891c252022ca3f.e9be1c226b4184298a60d5e1076cd600, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1683694526577 2023-05-10 04:55:27,844 DEBUG [RS:2;jenkins-hbase4:37823-longCompactions-0] compactions.Compactor(207): Compacting 19631cfae8b94f0e9b5788c3a22fd03b.e9be1c226b4184298a60d5e1076cd600, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1683694526743 2023-05-10 04:55:27,844 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=147 updating hbase:meta row=48970235bef33aa5251ac68151d7d8b4, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:27,845 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegion,,1683694526936.48970235bef33aa5251ac68151d7d8b4.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1683694527844"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1683694527844"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1683694527844"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683694527844"}]},"ts":"1683694527844"} 2023-05-10 04:55:27,845 DEBUG [RS:2;jenkins-hbase4:37823-longCompactions-0] compactions.Compactor(207): Compacting 30a77dd573b64ac598ec88e3afa961e4.e9be1c226b4184298a60d5e1076cd600, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1683694526929 2023-05-10 04:55:27,845 DEBUG [RS:2;jenkins-hbase4:37823-shortCompactions-0] compactions.Compactor(207): Compacting 46158d7d3eea4447ab610c5263cb7a20.e9be1c226b4184298a60d5e1076cd600, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1683694526743 2023-05-10 04:55:27,846 DEBUG [RS:2;jenkins-hbase4:37823-shortCompactions-0] compactions.Compactor(207): Compacting e7a24003dc044b82b9d96bae6d63b040.e9be1c226b4184298a60d5e1076cd600, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1683694526929 2023-05-10 04:55:27,847 INFO [StoreOpener-c5e0f2bc1f34b2b275d44dfbd485b943-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region c5e0f2bc1f34b2b275d44dfbd485b943 2023-05-10 04:55:27,847 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=149, resume processing ppid=147 2023-05-10 04:55:27,848 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=149, ppid=147, state=SUCCESS; OpenRegionProcedure 48970235bef33aa5251ac68151d7d8b4, server=jenkins-hbase4.apache.org,37823,1683694419534 in 239 msec 2023-05-10 04:55:27,848 DEBUG [StoreOpener-c5e0f2bc1f34b2b275d44dfbd485b943-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/c5e0f2bc1f34b2b275d44dfbd485b943/cf1 2023-05-10 04:55:27,848 DEBUG [StoreOpener-c5e0f2bc1f34b2b275d44dfbd485b943-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/c5e0f2bc1f34b2b275d44dfbd485b943/cf1 2023-05-10 04:55:27,849 INFO [StoreOpener-c5e0f2bc1f34b2b275d44dfbd485b943-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c5e0f2bc1f34b2b275d44dfbd485b943 columnFamilyName cf1 2023-05-10 04:55:27,849 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=147, ppid=144, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=48970235bef33aa5251ac68151d7d8b4, ASSIGN in 396 msec 2023-05-10 04:55:27,858 INFO [RS:2;jenkins-hbase4:37823-shortCompactions-0] throttle.PressureAwareThroughputController(145): 48970235bef33aa5251ac68151d7d8b4#cf1#compaction#140 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-05-10 04:55:27,863 INFO [RS:2;jenkins-hbase4:37823-longCompactions-0] throttle.PressureAwareThroughputController(145): 48970235bef33aa5251ac68151d7d8b4#cf2#compaction#141 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-05-10 04:55:27,870 DEBUG [StoreOpener-c5e0f2bc1f34b2b275d44dfbd485b943-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/c5e0f2bc1f34b2b275d44dfbd485b943/cf1/34e676aeee2c4595a9dc599ec25aaf43.e9be1c226b4184298a60d5e1076cd600->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf1/34e676aeee2c4595a9dc599ec25aaf43-top 2023-05-10 04:55:27,872 DEBUG [StoreOpener-c5e0f2bc1f34b2b275d44dfbd485b943-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/c5e0f2bc1f34b2b275d44dfbd485b943/cf1/46158d7d3eea4447ab610c5263cb7a20.e9be1c226b4184298a60d5e1076cd600->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf1/46158d7d3eea4447ab610c5263cb7a20-top 2023-05-10 04:55:27,876 DEBUG [StoreOpener-c5e0f2bc1f34b2b275d44dfbd485b943-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/c5e0f2bc1f34b2b275d44dfbd485b943/cf1/58d433c9ed254876ac1bfa1b33856169.e9be1c226b4184298a60d5e1076cd600->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf1/58d433c9ed254876ac1bfa1b33856169-top 2023-05-10 04:55:27,878 DEBUG [RS:2;jenkins-hbase4:37823-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/48970235bef33aa5251ac68151d7d8b4/.tmp/cf2/bbb96d1f7d1b469788a061b27a1dc5ea as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/48970235bef33aa5251ac68151d7d8b4/cf2/bbb96d1f7d1b469788a061b27a1dc5ea 2023-05-10 04:55:27,879 DEBUG [StoreOpener-c5e0f2bc1f34b2b275d44dfbd485b943-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/c5e0f2bc1f34b2b275d44dfbd485b943/cf1/91101e4c2e2e42578d390290caf782d5.e9be1c226b4184298a60d5e1076cd600->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf1/91101e4c2e2e42578d390290caf782d5-top 2023-05-10 04:55:27,882 INFO [RS:2;jenkins-hbase4:37823-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 48970235bef33aa5251ac68151d7d8b4/cf2 of 48970235bef33aa5251ac68151d7d8b4 into bbb96d1f7d1b469788a061b27a1dc5ea(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-05-10 04:55:27,882 DEBUG [RS:2;jenkins-hbase4:37823-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 48970235bef33aa5251ac68151d7d8b4: 2023-05-10 04:55:27,882 INFO [RS:2;jenkins-hbase4:37823-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,,1683694526936.48970235bef33aa5251ac68151d7d8b4., storeName=48970235bef33aa5251ac68151d7d8b4/cf2, priority=9, startTime=1683694527839; duration=0sec 2023-05-10 04:55:27,883 DEBUG [RS:2;jenkins-hbase4:37823-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-10 04:55:27,889 DEBUG [StoreOpener-c5e0f2bc1f34b2b275d44dfbd485b943-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/c5e0f2bc1f34b2b275d44dfbd485b943/cf1/af98ce1ca36546c289891c252022ca3f.e9be1c226b4184298a60d5e1076cd600->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf1/af98ce1ca36546c289891c252022ca3f-top 2023-05-10 04:55:27,900 DEBUG [StoreOpener-c5e0f2bc1f34b2b275d44dfbd485b943-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/c5e0f2bc1f34b2b275d44dfbd485b943/cf1/e7a24003dc044b82b9d96bae6d63b040.e9be1c226b4184298a60d5e1076cd600->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf1/e7a24003dc044b82b9d96bae6d63b040-top 2023-05-10 04:55:27,903 DEBUG [StoreOpener-c5e0f2bc1f34b2b275d44dfbd485b943-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/c5e0f2bc1f34b2b275d44dfbd485b943/cf1/ec03059b17284629846b089825945254.e9be1c226b4184298a60d5e1076cd600->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf1/ec03059b17284629846b089825945254-top 2023-05-10 04:55:27,903 INFO [StoreOpener-c5e0f2bc1f34b2b275d44dfbd485b943-1] regionserver.HStore(310): Store=c5e0f2bc1f34b2b275d44dfbd485b943/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-10 04:55:27,903 INFO [StoreOpener-c5e0f2bc1f34b2b275d44dfbd485b943-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region c5e0f2bc1f34b2b275d44dfbd485b943 2023-05-10 04:55:27,904 DEBUG [StoreOpener-c5e0f2bc1f34b2b275d44dfbd485b943-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/c5e0f2bc1f34b2b275d44dfbd485b943/cf2 2023-05-10 04:55:27,904 DEBUG [StoreOpener-c5e0f2bc1f34b2b275d44dfbd485b943-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/c5e0f2bc1f34b2b275d44dfbd485b943/cf2 2023-05-10 04:55:27,904 INFO [StoreOpener-c5e0f2bc1f34b2b275d44dfbd485b943-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c5e0f2bc1f34b2b275d44dfbd485b943 columnFamilyName cf2 2023-05-10 04:55:27,915 DEBUG [StoreOpener-c5e0f2bc1f34b2b275d44dfbd485b943-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/c5e0f2bc1f34b2b275d44dfbd485b943/cf2/19631cfae8b94f0e9b5788c3a22fd03b.e9be1c226b4184298a60d5e1076cd600->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf2/19631cfae8b94f0e9b5788c3a22fd03b-top 2023-05-10 04:55:27,918 DEBUG [StoreOpener-c5e0f2bc1f34b2b275d44dfbd485b943-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/c5e0f2bc1f34b2b275d44dfbd485b943/cf2/2cc0f196bb124e2eaab06768c81709b6.e9be1c226b4184298a60d5e1076cd600->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf2/2cc0f196bb124e2eaab06768c81709b6-top 2023-05-10 04:55:27,921 DEBUG [StoreOpener-c5e0f2bc1f34b2b275d44dfbd485b943-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/c5e0f2bc1f34b2b275d44dfbd485b943/cf2/30a77dd573b64ac598ec88e3afa961e4.e9be1c226b4184298a60d5e1076cd600->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf2/30a77dd573b64ac598ec88e3afa961e4-top 2023-05-10 04:55:27,923 DEBUG [StoreOpener-c5e0f2bc1f34b2b275d44dfbd485b943-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/c5e0f2bc1f34b2b275d44dfbd485b943/cf2/4664a731a30c46e2bf2e8bad410a89bd.e9be1c226b4184298a60d5e1076cd600->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf2/4664a731a30c46e2bf2e8bad410a89bd-top 2023-05-10 04:55:27,926 DEBUG [StoreOpener-c5e0f2bc1f34b2b275d44dfbd485b943-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/c5e0f2bc1f34b2b275d44dfbd485b943/cf2/57e73fe7c40143d6a3a6e5618f59ed0a.e9be1c226b4184298a60d5e1076cd600->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf2/57e73fe7c40143d6a3a6e5618f59ed0a-top 2023-05-10 04:55:27,928 DEBUG [StoreOpener-c5e0f2bc1f34b2b275d44dfbd485b943-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/c5e0f2bc1f34b2b275d44dfbd485b943/cf2/dd83d4fecabb4f31af903d8b1e8934b8.e9be1c226b4184298a60d5e1076cd600->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf2/dd83d4fecabb4f31af903d8b1e8934b8-top 2023-05-10 04:55:27,931 DEBUG [StoreOpener-c5e0f2bc1f34b2b275d44dfbd485b943-1] regionserver.HStore(539): loaded hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/c5e0f2bc1f34b2b275d44dfbd485b943/cf2/e29767cd1918445bbad6143229520f9e.e9be1c226b4184298a60d5e1076cd600->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf2/e29767cd1918445bbad6143229520f9e-top 2023-05-10 04:55:27,931 INFO [StoreOpener-c5e0f2bc1f34b2b275d44dfbd485b943-1] regionserver.HStore(310): Store=c5e0f2bc1f34b2b275d44dfbd485b943/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-05-10 04:55:27,931 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/c5e0f2bc1f34b2b275d44dfbd485b943 2023-05-10 04:55:27,932 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/c5e0f2bc1f34b2b275d44dfbd485b943 2023-05-10 04:55:27,934 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegion descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-05-10 04:55:27,935 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1055): writing seq id for c5e0f2bc1f34b2b275d44dfbd485b943 2023-05-10 04:55:27,935 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1072): Opened c5e0f2bc1f34b2b275d44dfbd485b943; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=64703204, jitterRate=-0.03584712743759155}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-05-10 04:55:27,935 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(965): Region open journal for c5e0f2bc1f34b2b275d44dfbd485b943: 2023-05-10 04:55:27,936 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2334): Post open deploy tasks for testSplitTableRegion,41,1683694526936.c5e0f2bc1f34b2b275d44dfbd485b943., pid=150, masterSystemTime=1683694527758 2023-05-10 04:55:27,936 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-10 04:55:27,937 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-05-10 04:55:27,937 DEBUG [RS:2;jenkins-hbase4:37823-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-05-10 04:55:27,937 INFO [RS:2;jenkins-hbase4:37823-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegion,41,1683694526936.c5e0f2bc1f34b2b275d44dfbd485b943. 2023-05-10 04:55:27,937 DEBUG [RS:2;jenkins-hbase4:37823-longCompactions-0] regionserver.HStore(1912): c5e0f2bc1f34b2b275d44dfbd485b943/cf1 is initiating minor compaction (all files) 2023-05-10 04:55:27,937 INFO [RS:2;jenkins-hbase4:37823-longCompactions-0] regionserver.HRegion(2259): Starting compaction of c5e0f2bc1f34b2b275d44dfbd485b943/cf1 in testSplitTableRegion,41,1683694526936.c5e0f2bc1f34b2b275d44dfbd485b943. 2023-05-10 04:55:27,937 INFO [RS:2;jenkins-hbase4:37823-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/c5e0f2bc1f34b2b275d44dfbd485b943/cf1/58d433c9ed254876ac1bfa1b33856169.e9be1c226b4184298a60d5e1076cd600->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf1/58d433c9ed254876ac1bfa1b33856169-top, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/c5e0f2bc1f34b2b275d44dfbd485b943/cf1/34e676aeee2c4595a9dc599ec25aaf43.e9be1c226b4184298a60d5e1076cd600->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf1/34e676aeee2c4595a9dc599ec25aaf43-top, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/c5e0f2bc1f34b2b275d44dfbd485b943/cf1/ec03059b17284629846b089825945254.e9be1c226b4184298a60d5e1076cd600->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf1/ec03059b17284629846b089825945254-top, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/c5e0f2bc1f34b2b275d44dfbd485b943/cf1/91101e4c2e2e42578d390290caf782d5.e9be1c226b4184298a60d5e1076cd600->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf1/91101e4c2e2e42578d390290caf782d5-top, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/c5e0f2bc1f34b2b275d44dfbd485b943/cf1/af98ce1ca36546c289891c252022ca3f.e9be1c226b4184298a60d5e1076cd600->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf1/af98ce1ca36546c289891c252022ca3f-top, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/c5e0f2bc1f34b2b275d44dfbd485b943/cf1/46158d7d3eea4447ab610c5263cb7a20.e9be1c226b4184298a60d5e1076cd600->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf1/46158d7d3eea4447ab610c5263cb7a20-top, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/c5e0f2bc1f34b2b275d44dfbd485b943/cf1/e7a24003dc044b82b9d96bae6d63b040.e9be1c226b4184298a60d5e1076cd600->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf1/e7a24003dc044b82b9d96bae6d63b040-top] into tmpdir=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/c5e0f2bc1f34b2b275d44dfbd485b943/.tmp, totalSize=34.9 K 2023-05-10 04:55:27,938 DEBUG [RS:2;jenkins-hbase4:37823-longCompactions-0] compactions.Compactor(207): Compacting 58d433c9ed254876ac1bfa1b33856169.e9be1c226b4184298a60d5e1076cd600, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1683694525880 2023-05-10 04:55:27,938 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionServer(2361): Finished post open deploy task for testSplitTableRegion,41,1683694526936.c5e0f2bc1f34b2b275d44dfbd485b943. 2023-05-10 04:55:27,938 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase4:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegion,41,1683694526936.c5e0f2bc1f34b2b275d44dfbd485b943. 2023-05-10 04:55:27,938 DEBUG [RS:2;jenkins-hbase4:37823-longCompactions-0] compactions.Compactor(207): Compacting 34e676aeee2c4595a9dc599ec25aaf43.e9be1c226b4184298a60d5e1076cd600, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1683694526050 2023-05-10 04:55:27,939 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=148 updating hbase:meta row=c5e0f2bc1f34b2b275d44dfbd485b943, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:27,939 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegion,41,1683694526936.c5e0f2bc1f34b2b275d44dfbd485b943.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1683694527938"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1683694527938"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1683694527938"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1683694527938"}]},"ts":"1683694527938"} 2023-05-10 04:55:27,939 DEBUG [RS:2;jenkins-hbase4:37823-longCompactions-0] compactions.Compactor(207): Compacting ec03059b17284629846b089825945254.e9be1c226b4184298a60d5e1076cd600, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1683694526223 2023-05-10 04:55:27,939 DEBUG [RS:2;jenkins-hbase4:37823-longCompactions-0] compactions.Compactor(207): Compacting 91101e4c2e2e42578d390290caf782d5.e9be1c226b4184298a60d5e1076cd600, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1683694526398 2023-05-10 04:55:27,940 DEBUG [RS:2;jenkins-hbase4:37823-longCompactions-0] compactions.Compactor(207): Compacting af98ce1ca36546c289891c252022ca3f.e9be1c226b4184298a60d5e1076cd600, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1683694526577 2023-05-10 04:55:27,940 DEBUG [RS:2;jenkins-hbase4:37823-longCompactions-0] compactions.Compactor(207): Compacting 46158d7d3eea4447ab610c5263cb7a20.e9be1c226b4184298a60d5e1076cd600, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1683694526743 2023-05-10 04:55:27,940 DEBUG [RS:2;jenkins-hbase4:37823-longCompactions-0] compactions.Compactor(207): Compacting e7a24003dc044b82b9d96bae6d63b040.e9be1c226b4184298a60d5e1076cd600, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1683694526929 2023-05-10 04:55:27,941 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=150, resume processing ppid=148 2023-05-10 04:55:27,941 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=150, ppid=148, state=SUCCESS; OpenRegionProcedure c5e0f2bc1f34b2b275d44dfbd485b943, server=jenkins-hbase4.apache.org,37823,1683694419534 in 330 msec 2023-05-10 04:55:27,942 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=148, resume processing ppid=144 2023-05-10 04:55:27,942 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=148, ppid=144, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=c5e0f2bc1f34b2b275d44dfbd485b943, ASSIGN in 489 msec 2023-05-10 04:55:27,944 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=144, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegion, parent=e9be1c226b4184298a60d5e1076cd600, daughterA=48970235bef33aa5251ac68151d7d8b4, daughterB=c5e0f2bc1f34b2b275d44dfbd485b943 in 1.0070 sec 2023-05-10 04:55:27,948 INFO [RS:2;jenkins-hbase4:37823-longCompactions-0] throttle.PressureAwareThroughputController(145): c5e0f2bc1f34b2b275d44dfbd485b943#cf1#compaction#142 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-05-10 04:55:27,958 DEBUG [RS:2;jenkins-hbase4:37823-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/c5e0f2bc1f34b2b275d44dfbd485b943/.tmp/cf1/557e0ebd8b0d4befa4afdf90e2f539b2 as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/c5e0f2bc1f34b2b275d44dfbd485b943/cf1/557e0ebd8b0d4befa4afdf90e2f539b2 2023-05-10 04:55:27,962 INFO [RS:2;jenkins-hbase4:37823-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in c5e0f2bc1f34b2b275d44dfbd485b943/cf1 of c5e0f2bc1f34b2b275d44dfbd485b943 into 557e0ebd8b0d4befa4afdf90e2f539b2(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-05-10 04:55:27,962 DEBUG [RS:2;jenkins-hbase4:37823-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for c5e0f2bc1f34b2b275d44dfbd485b943: 2023-05-10 04:55:27,962 INFO [RS:2;jenkins-hbase4:37823-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,41,1683694526936.c5e0f2bc1f34b2b275d44dfbd485b943., storeName=c5e0f2bc1f34b2b275d44dfbd485b943/cf1, priority=9, startTime=1683694527936; duration=0sec 2023-05-10 04:55:27,962 DEBUG [RS:2;jenkins-hbase4:37823-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-05-10 04:55:27,962 DEBUG [RS:2;jenkins-hbase4:37823-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-05-10 04:55:27,962 INFO [RS:2;jenkins-hbase4:37823-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegion,41,1683694526936.c5e0f2bc1f34b2b275d44dfbd485b943. 2023-05-10 04:55:27,962 DEBUG [RS:2;jenkins-hbase4:37823-longCompactions-0] regionserver.HStore(1912): c5e0f2bc1f34b2b275d44dfbd485b943/cf2 is initiating minor compaction (all files) 2023-05-10 04:55:27,962 INFO [RS:2;jenkins-hbase4:37823-longCompactions-0] regionserver.HRegion(2259): Starting compaction of c5e0f2bc1f34b2b275d44dfbd485b943/cf2 in testSplitTableRegion,41,1683694526936.c5e0f2bc1f34b2b275d44dfbd485b943. 2023-05-10 04:55:27,963 INFO [RS:2;jenkins-hbase4:37823-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/c5e0f2bc1f34b2b275d44dfbd485b943/cf2/4664a731a30c46e2bf2e8bad410a89bd.e9be1c226b4184298a60d5e1076cd600->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf2/4664a731a30c46e2bf2e8bad410a89bd-top, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/c5e0f2bc1f34b2b275d44dfbd485b943/cf2/e29767cd1918445bbad6143229520f9e.e9be1c226b4184298a60d5e1076cd600->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf2/e29767cd1918445bbad6143229520f9e-top, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/c5e0f2bc1f34b2b275d44dfbd485b943/cf2/dd83d4fecabb4f31af903d8b1e8934b8.e9be1c226b4184298a60d5e1076cd600->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf2/dd83d4fecabb4f31af903d8b1e8934b8-top, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/c5e0f2bc1f34b2b275d44dfbd485b943/cf2/57e73fe7c40143d6a3a6e5618f59ed0a.e9be1c226b4184298a60d5e1076cd600->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf2/57e73fe7c40143d6a3a6e5618f59ed0a-top, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/c5e0f2bc1f34b2b275d44dfbd485b943/cf2/2cc0f196bb124e2eaab06768c81709b6.e9be1c226b4184298a60d5e1076cd600->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf2/2cc0f196bb124e2eaab06768c81709b6-top, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/c5e0f2bc1f34b2b275d44dfbd485b943/cf2/19631cfae8b94f0e9b5788c3a22fd03b.e9be1c226b4184298a60d5e1076cd600->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf2/19631cfae8b94f0e9b5788c3a22fd03b-top, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/c5e0f2bc1f34b2b275d44dfbd485b943/cf2/30a77dd573b64ac598ec88e3afa961e4.e9be1c226b4184298a60d5e1076cd600->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf2/30a77dd573b64ac598ec88e3afa961e4-top] into tmpdir=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/c5e0f2bc1f34b2b275d44dfbd485b943/.tmp, totalSize=34.9 K 2023-05-10 04:55:27,963 DEBUG [RS:2;jenkins-hbase4:37823-longCompactions-0] compactions.Compactor(207): Compacting 4664a731a30c46e2bf2e8bad410a89bd.e9be1c226b4184298a60d5e1076cd600, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1683694525880 2023-05-10 04:55:27,963 DEBUG [RS:2;jenkins-hbase4:37823-longCompactions-0] compactions.Compactor(207): Compacting e29767cd1918445bbad6143229520f9e.e9be1c226b4184298a60d5e1076cd600, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1683694526050 2023-05-10 04:55:27,964 DEBUG [RS:2;jenkins-hbase4:37823-longCompactions-0] compactions.Compactor(207): Compacting dd83d4fecabb4f31af903d8b1e8934b8.e9be1c226b4184298a60d5e1076cd600, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1683694526223 2023-05-10 04:55:27,964 DEBUG [RS:2;jenkins-hbase4:37823-longCompactions-0] compactions.Compactor(207): Compacting 57e73fe7c40143d6a3a6e5618f59ed0a.e9be1c226b4184298a60d5e1076cd600, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1683694526398 2023-05-10 04:55:27,964 DEBUG [RS:2;jenkins-hbase4:37823-longCompactions-0] compactions.Compactor(207): Compacting 2cc0f196bb124e2eaab06768c81709b6.e9be1c226b4184298a60d5e1076cd600, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1683694526577 2023-05-10 04:55:27,965 DEBUG [RS:2;jenkins-hbase4:37823-longCompactions-0] compactions.Compactor(207): Compacting 19631cfae8b94f0e9b5788c3a22fd03b.e9be1c226b4184298a60d5e1076cd600, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1683694526743 2023-05-10 04:55:27,965 DEBUG [RS:2;jenkins-hbase4:37823-longCompactions-0] compactions.Compactor(207): Compacting 30a77dd573b64ac598ec88e3afa961e4.e9be1c226b4184298a60d5e1076cd600, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1683694526929 2023-05-10 04:55:27,972 INFO [RS:2;jenkins-hbase4:37823-longCompactions-0] throttle.PressureAwareThroughputController(145): c5e0f2bc1f34b2b275d44dfbd485b943#cf2#compaction#143 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-05-10 04:55:27,984 DEBUG [RS:2;jenkins-hbase4:37823-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/c5e0f2bc1f34b2b275d44dfbd485b943/.tmp/cf2/996e883a0cee4591ac08cfa7ae1e9935 as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/c5e0f2bc1f34b2b275d44dfbd485b943/cf2/996e883a0cee4591ac08cfa7ae1e9935 2023-05-10 04:55:27,988 INFO [RS:2;jenkins-hbase4:37823-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in c5e0f2bc1f34b2b275d44dfbd485b943/cf2 of c5e0f2bc1f34b2b275d44dfbd485b943 into 996e883a0cee4591ac08cfa7ae1e9935(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-05-10 04:55:27,988 DEBUG [RS:2;jenkins-hbase4:37823-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for c5e0f2bc1f34b2b275d44dfbd485b943: 2023-05-10 04:55:27,988 INFO [RS:2;jenkins-hbase4:37823-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,41,1683694526936.c5e0f2bc1f34b2b275d44dfbd485b943., storeName=c5e0f2bc1f34b2b275d44dfbd485b943/cf2, priority=9, startTime=1683694527937; duration=0sec 2023-05-10 04:55:27,989 DEBUG [RS:2;jenkins-hbase4:37823-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-10 04:55:28,188 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=37823] ipc.CallRunner(144): callId: 1148 service: ClientService methodName: Scan size: 143 connection: 172.31.14.131:55416 deadline: 1683694588188, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitTableRegion,,1683694525376.e9be1c226b4184298a60d5e1076cd600. is not online on jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:28,270 DEBUG [RS:2;jenkins-hbase4:37823-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/48970235bef33aa5251ac68151d7d8b4/.tmp/cf1/643c74205d064793bf31397cd72711db as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/48970235bef33aa5251ac68151d7d8b4/cf1/643c74205d064793bf31397cd72711db 2023-05-10 04:55:28,274 INFO [RS:2;jenkins-hbase4:37823-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 48970235bef33aa5251ac68151d7d8b4/cf1 of 48970235bef33aa5251ac68151d7d8b4 into 643c74205d064793bf31397cd72711db(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-05-10 04:55:28,274 DEBUG [RS:2;jenkins-hbase4:37823-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 48970235bef33aa5251ac68151d7d8b4: 2023-05-10 04:55:28,274 INFO [RS:2;jenkins-hbase4:37823-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,,1683694526936.48970235bef33aa5251ac68151d7d8b4., storeName=48970235bef33aa5251ac68151d7d8b4/cf1, priority=9, startTime=1683694527838; duration=0sec 2023-05-10 04:55:28,274 DEBUG [RS:2;jenkins-hbase4:37823-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-05-10 04:55:28,300 DEBUG [Listener at localhost/45007] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-05-10 04:55:28,304 INFO [Listener at localhost/45007] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-05-10 04:55:28,305 DEBUG [Listener at localhost/45007] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-05-10 04:55:28,314 WARN [Listener at localhost/45007] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-05-10 04:55:28,315 INFO [Listener at localhost/45007] client.HBaseAdmin$15(890): Started disable of testSplitTableRegion 2023-05-10 04:55:28,315 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.HMaster$11(2418): Client=jenkins//172.31.14.131 disable testSplitTableRegion 2023-05-10 04:55:28,316 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] procedure2.ProcedureExecutor(1029): Stored pid=151, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegion 2023-05-10 04:55:28,318 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(1227): Checking to see if procedure is done pid=151 2023-05-10 04:55:28,319 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683694528319"}]},"ts":"1683694528319"} 2023-05-10 04:55:28,320 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=DISABLING in hbase:meta 2023-05-10 04:55:28,322 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegion to state=DISABLING 2023-05-10 04:55:28,323 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=152, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=48970235bef33aa5251ac68151d7d8b4, UNASSIGN}, {pid=153, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=c5e0f2bc1f34b2b275d44dfbd485b943, UNASSIGN}] 2023-05-10 04:55:28,323 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=153, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=c5e0f2bc1f34b2b275d44dfbd485b943, UNASSIGN 2023-05-10 04:55:28,324 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=153 updating hbase:meta row=c5e0f2bc1f34b2b275d44dfbd485b943, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:28,324 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,41,1683694526936.c5e0f2bc1f34b2b275d44dfbd485b943.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1683694528324"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683694528324"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683694528324"}]},"ts":"1683694528324"} 2023-05-10 04:55:28,325 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=154, ppid=153, state=RUNNABLE; CloseRegionProcedure c5e0f2bc1f34b2b275d44dfbd485b943, server=jenkins-hbase4.apache.org,37823,1683694419534}] 2023-05-10 04:55:28,327 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=152, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=48970235bef33aa5251ac68151d7d8b4, UNASSIGN 2023-05-10 04:55:28,327 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=152 updating hbase:meta row=48970235bef33aa5251ac68151d7d8b4, regionState=CLOSING, regionLocation=jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:28,327 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1683694526936.48970235bef33aa5251ac68151d7d8b4.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1683694528327"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1683694528327"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1683694528327"}]},"ts":"1683694528327"} 2023-05-10 04:55:28,331 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=155, ppid=152, state=RUNNABLE; CloseRegionProcedure 48970235bef33aa5251ac68151d7d8b4, server=jenkins-hbase4.apache.org,37823,1683694419534}] 2023-05-10 04:55:28,419 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(1227): Checking to see if procedure is done pid=151 2023-05-10 04:55:28,477 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close c5e0f2bc1f34b2b275d44dfbd485b943 2023-05-10 04:55:28,477 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing c5e0f2bc1f34b2b275d44dfbd485b943, disabling compactions & flushes 2023-05-10 04:55:28,477 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegion,41,1683694526936.c5e0f2bc1f34b2b275d44dfbd485b943. 2023-05-10 04:55:28,477 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegion,41,1683694526936.c5e0f2bc1f34b2b275d44dfbd485b943. 2023-05-10 04:55:28,477 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegion,41,1683694526936.c5e0f2bc1f34b2b275d44dfbd485b943. after waiting 0 ms 2023-05-10 04:55:28,477 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegion,41,1683694526936.c5e0f2bc1f34b2b275d44dfbd485b943. 2023-05-10 04:55:28,478 DEBUG [StoreCloser-testSplitTableRegion,41,1683694526936.c5e0f2bc1f34b2b275d44dfbd485b943.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/c5e0f2bc1f34b2b275d44dfbd485b943/cf1/58d433c9ed254876ac1bfa1b33856169.e9be1c226b4184298a60d5e1076cd600->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf1/58d433c9ed254876ac1bfa1b33856169-top, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/c5e0f2bc1f34b2b275d44dfbd485b943/cf1/34e676aeee2c4595a9dc599ec25aaf43.e9be1c226b4184298a60d5e1076cd600->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf1/34e676aeee2c4595a9dc599ec25aaf43-top, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/c5e0f2bc1f34b2b275d44dfbd485b943/cf1/ec03059b17284629846b089825945254.e9be1c226b4184298a60d5e1076cd600->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf1/ec03059b17284629846b089825945254-top, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/c5e0f2bc1f34b2b275d44dfbd485b943/cf1/91101e4c2e2e42578d390290caf782d5.e9be1c226b4184298a60d5e1076cd600->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf1/91101e4c2e2e42578d390290caf782d5-top, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/c5e0f2bc1f34b2b275d44dfbd485b943/cf1/af98ce1ca36546c289891c252022ca3f.e9be1c226b4184298a60d5e1076cd600->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf1/af98ce1ca36546c289891c252022ca3f-top, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/c5e0f2bc1f34b2b275d44dfbd485b943/cf1/46158d7d3eea4447ab610c5263cb7a20.e9be1c226b4184298a60d5e1076cd600->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf1/46158d7d3eea4447ab610c5263cb7a20-top, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/c5e0f2bc1f34b2b275d44dfbd485b943/cf1/e7a24003dc044b82b9d96bae6d63b040.e9be1c226b4184298a60d5e1076cd600->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf1/e7a24003dc044b82b9d96bae6d63b040-top] to archive 2023-05-10 04:55:28,478 DEBUG [StoreCloser-testSplitTableRegion,41,1683694526936.c5e0f2bc1f34b2b275d44dfbd485b943.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-05-10 04:55:28,480 DEBUG [StoreCloser-testSplitTableRegion,41,1683694526936.c5e0f2bc1f34b2b275d44dfbd485b943.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/c5e0f2bc1f34b2b275d44dfbd485b943/cf1/58d433c9ed254876ac1bfa1b33856169.e9be1c226b4184298a60d5e1076cd600 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegion/c5e0f2bc1f34b2b275d44dfbd485b943/cf1/58d433c9ed254876ac1bfa1b33856169.e9be1c226b4184298a60d5e1076cd600 2023-05-10 04:55:28,481 DEBUG [StoreCloser-testSplitTableRegion,41,1683694526936.c5e0f2bc1f34b2b275d44dfbd485b943.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/c5e0f2bc1f34b2b275d44dfbd485b943/cf1/34e676aeee2c4595a9dc599ec25aaf43.e9be1c226b4184298a60d5e1076cd600 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegion/c5e0f2bc1f34b2b275d44dfbd485b943/cf1/34e676aeee2c4595a9dc599ec25aaf43.e9be1c226b4184298a60d5e1076cd600 2023-05-10 04:55:28,482 DEBUG [StoreCloser-testSplitTableRegion,41,1683694526936.c5e0f2bc1f34b2b275d44dfbd485b943.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/c5e0f2bc1f34b2b275d44dfbd485b943/cf1/ec03059b17284629846b089825945254.e9be1c226b4184298a60d5e1076cd600 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegion/c5e0f2bc1f34b2b275d44dfbd485b943/cf1/ec03059b17284629846b089825945254.e9be1c226b4184298a60d5e1076cd600 2023-05-10 04:55:28,483 DEBUG [StoreCloser-testSplitTableRegion,41,1683694526936.c5e0f2bc1f34b2b275d44dfbd485b943.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/c5e0f2bc1f34b2b275d44dfbd485b943/cf1/91101e4c2e2e42578d390290caf782d5.e9be1c226b4184298a60d5e1076cd600 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegion/c5e0f2bc1f34b2b275d44dfbd485b943/cf1/91101e4c2e2e42578d390290caf782d5.e9be1c226b4184298a60d5e1076cd600 2023-05-10 04:55:28,484 DEBUG [StoreCloser-testSplitTableRegion,41,1683694526936.c5e0f2bc1f34b2b275d44dfbd485b943.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/c5e0f2bc1f34b2b275d44dfbd485b943/cf1/af98ce1ca36546c289891c252022ca3f.e9be1c226b4184298a60d5e1076cd600 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegion/c5e0f2bc1f34b2b275d44dfbd485b943/cf1/af98ce1ca36546c289891c252022ca3f.e9be1c226b4184298a60d5e1076cd600 2023-05-10 04:55:28,485 DEBUG [StoreCloser-testSplitTableRegion,41,1683694526936.c5e0f2bc1f34b2b275d44dfbd485b943.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/c5e0f2bc1f34b2b275d44dfbd485b943/cf1/46158d7d3eea4447ab610c5263cb7a20.e9be1c226b4184298a60d5e1076cd600 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegion/c5e0f2bc1f34b2b275d44dfbd485b943/cf1/46158d7d3eea4447ab610c5263cb7a20.e9be1c226b4184298a60d5e1076cd600 2023-05-10 04:55:28,486 DEBUG [StoreCloser-testSplitTableRegion,41,1683694526936.c5e0f2bc1f34b2b275d44dfbd485b943.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/c5e0f2bc1f34b2b275d44dfbd485b943/cf1/e7a24003dc044b82b9d96bae6d63b040.e9be1c226b4184298a60d5e1076cd600 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegion/c5e0f2bc1f34b2b275d44dfbd485b943/cf1/e7a24003dc044b82b9d96bae6d63b040.e9be1c226b4184298a60d5e1076cd600 2023-05-10 04:55:28,487 DEBUG [StoreCloser-testSplitTableRegion,41,1683694526936.c5e0f2bc1f34b2b275d44dfbd485b943.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/c5e0f2bc1f34b2b275d44dfbd485b943/cf2/4664a731a30c46e2bf2e8bad410a89bd.e9be1c226b4184298a60d5e1076cd600->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf2/4664a731a30c46e2bf2e8bad410a89bd-top, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/c5e0f2bc1f34b2b275d44dfbd485b943/cf2/e29767cd1918445bbad6143229520f9e.e9be1c226b4184298a60d5e1076cd600->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf2/e29767cd1918445bbad6143229520f9e-top, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/c5e0f2bc1f34b2b275d44dfbd485b943/cf2/dd83d4fecabb4f31af903d8b1e8934b8.e9be1c226b4184298a60d5e1076cd600->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf2/dd83d4fecabb4f31af903d8b1e8934b8-top, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/c5e0f2bc1f34b2b275d44dfbd485b943/cf2/57e73fe7c40143d6a3a6e5618f59ed0a.e9be1c226b4184298a60d5e1076cd600->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf2/57e73fe7c40143d6a3a6e5618f59ed0a-top, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/c5e0f2bc1f34b2b275d44dfbd485b943/cf2/2cc0f196bb124e2eaab06768c81709b6.e9be1c226b4184298a60d5e1076cd600->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf2/2cc0f196bb124e2eaab06768c81709b6-top, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/c5e0f2bc1f34b2b275d44dfbd485b943/cf2/19631cfae8b94f0e9b5788c3a22fd03b.e9be1c226b4184298a60d5e1076cd600->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf2/19631cfae8b94f0e9b5788c3a22fd03b-top, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/c5e0f2bc1f34b2b275d44dfbd485b943/cf2/30a77dd573b64ac598ec88e3afa961e4.e9be1c226b4184298a60d5e1076cd600->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf2/30a77dd573b64ac598ec88e3afa961e4-top] to archive 2023-05-10 04:55:28,488 DEBUG [StoreCloser-testSplitTableRegion,41,1683694526936.c5e0f2bc1f34b2b275d44dfbd485b943.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-05-10 04:55:28,489 DEBUG [StoreCloser-testSplitTableRegion,41,1683694526936.c5e0f2bc1f34b2b275d44dfbd485b943.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/c5e0f2bc1f34b2b275d44dfbd485b943/cf2/4664a731a30c46e2bf2e8bad410a89bd.e9be1c226b4184298a60d5e1076cd600 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegion/c5e0f2bc1f34b2b275d44dfbd485b943/cf2/4664a731a30c46e2bf2e8bad410a89bd.e9be1c226b4184298a60d5e1076cd600 2023-05-10 04:55:28,490 DEBUG [StoreCloser-testSplitTableRegion,41,1683694526936.c5e0f2bc1f34b2b275d44dfbd485b943.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/c5e0f2bc1f34b2b275d44dfbd485b943/cf2/e29767cd1918445bbad6143229520f9e.e9be1c226b4184298a60d5e1076cd600 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegion/c5e0f2bc1f34b2b275d44dfbd485b943/cf2/e29767cd1918445bbad6143229520f9e.e9be1c226b4184298a60d5e1076cd600 2023-05-10 04:55:28,491 DEBUG [StoreCloser-testSplitTableRegion,41,1683694526936.c5e0f2bc1f34b2b275d44dfbd485b943.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/c5e0f2bc1f34b2b275d44dfbd485b943/cf2/dd83d4fecabb4f31af903d8b1e8934b8.e9be1c226b4184298a60d5e1076cd600 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegion/c5e0f2bc1f34b2b275d44dfbd485b943/cf2/dd83d4fecabb4f31af903d8b1e8934b8.e9be1c226b4184298a60d5e1076cd600 2023-05-10 04:55:28,492 DEBUG [StoreCloser-testSplitTableRegion,41,1683694526936.c5e0f2bc1f34b2b275d44dfbd485b943.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/c5e0f2bc1f34b2b275d44dfbd485b943/cf2/57e73fe7c40143d6a3a6e5618f59ed0a.e9be1c226b4184298a60d5e1076cd600 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegion/c5e0f2bc1f34b2b275d44dfbd485b943/cf2/57e73fe7c40143d6a3a6e5618f59ed0a.e9be1c226b4184298a60d5e1076cd600 2023-05-10 04:55:28,493 DEBUG [StoreCloser-testSplitTableRegion,41,1683694526936.c5e0f2bc1f34b2b275d44dfbd485b943.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/c5e0f2bc1f34b2b275d44dfbd485b943/cf2/2cc0f196bb124e2eaab06768c81709b6.e9be1c226b4184298a60d5e1076cd600 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegion/c5e0f2bc1f34b2b275d44dfbd485b943/cf2/2cc0f196bb124e2eaab06768c81709b6.e9be1c226b4184298a60d5e1076cd600 2023-05-10 04:55:28,494 DEBUG [StoreCloser-testSplitTableRegion,41,1683694526936.c5e0f2bc1f34b2b275d44dfbd485b943.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/c5e0f2bc1f34b2b275d44dfbd485b943/cf2/19631cfae8b94f0e9b5788c3a22fd03b.e9be1c226b4184298a60d5e1076cd600 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegion/c5e0f2bc1f34b2b275d44dfbd485b943/cf2/19631cfae8b94f0e9b5788c3a22fd03b.e9be1c226b4184298a60d5e1076cd600 2023-05-10 04:55:28,494 DEBUG [StoreCloser-testSplitTableRegion,41,1683694526936.c5e0f2bc1f34b2b275d44dfbd485b943.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/c5e0f2bc1f34b2b275d44dfbd485b943/cf2/30a77dd573b64ac598ec88e3afa961e4.e9be1c226b4184298a60d5e1076cd600 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegion/c5e0f2bc1f34b2b275d44dfbd485b943/cf2/30a77dd573b64ac598ec88e3afa961e4.e9be1c226b4184298a60d5e1076cd600 2023-05-10 04:55:28,497 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/c5e0f2bc1f34b2b275d44dfbd485b943/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-05-10 04:55:28,498 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-10 04:55:28,498 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegion,41,1683694526936.c5e0f2bc1f34b2b275d44dfbd485b943. 2023-05-10 04:55:28,498 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for c5e0f2bc1f34b2b275d44dfbd485b943: 2023-05-10 04:55:28,499 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed c5e0f2bc1f34b2b275d44dfbd485b943 2023-05-10 04:55:28,499 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(111): Close 48970235bef33aa5251ac68151d7d8b4 2023-05-10 04:55:28,499 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 48970235bef33aa5251ac68151d7d8b4, disabling compactions & flushes 2023-05-10 04:55:28,499 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegion,,1683694526936.48970235bef33aa5251ac68151d7d8b4. 2023-05-10 04:55:28,499 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegion,,1683694526936.48970235bef33aa5251ac68151d7d8b4. 2023-05-10 04:55:28,499 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegion,,1683694526936.48970235bef33aa5251ac68151d7d8b4. after waiting 0 ms 2023-05-10 04:55:28,499 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegion,,1683694526936.48970235bef33aa5251ac68151d7d8b4. 2023-05-10 04:55:28,499 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=153 updating hbase:meta row=c5e0f2bc1f34b2b275d44dfbd485b943, regionState=CLOSED 2023-05-10 04:55:28,500 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegion,41,1683694526936.c5e0f2bc1f34b2b275d44dfbd485b943.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1683694528499"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683694528499"}]},"ts":"1683694528499"} 2023-05-10 04:55:28,501 DEBUG [StoreCloser-testSplitTableRegion,,1683694526936.48970235bef33aa5251ac68151d7d8b4.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/48970235bef33aa5251ac68151d7d8b4/cf1/58d433c9ed254876ac1bfa1b33856169.e9be1c226b4184298a60d5e1076cd600->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf1/58d433c9ed254876ac1bfa1b33856169-bottom, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/48970235bef33aa5251ac68151d7d8b4/cf1/34e676aeee2c4595a9dc599ec25aaf43.e9be1c226b4184298a60d5e1076cd600->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf1/34e676aeee2c4595a9dc599ec25aaf43-bottom, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/48970235bef33aa5251ac68151d7d8b4/cf1/ec03059b17284629846b089825945254.e9be1c226b4184298a60d5e1076cd600->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf1/ec03059b17284629846b089825945254-bottom, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/48970235bef33aa5251ac68151d7d8b4/cf1/91101e4c2e2e42578d390290caf782d5.e9be1c226b4184298a60d5e1076cd600->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf1/91101e4c2e2e42578d390290caf782d5-bottom, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/48970235bef33aa5251ac68151d7d8b4/cf1/af98ce1ca36546c289891c252022ca3f.e9be1c226b4184298a60d5e1076cd600->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf1/af98ce1ca36546c289891c252022ca3f-bottom, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/48970235bef33aa5251ac68151d7d8b4/cf1/46158d7d3eea4447ab610c5263cb7a20.e9be1c226b4184298a60d5e1076cd600->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf1/46158d7d3eea4447ab610c5263cb7a20-bottom, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/48970235bef33aa5251ac68151d7d8b4/cf1/e7a24003dc044b82b9d96bae6d63b040.e9be1c226b4184298a60d5e1076cd600->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf1/e7a24003dc044b82b9d96bae6d63b040-bottom] to archive 2023-05-10 04:55:28,501 DEBUG [StoreCloser-testSplitTableRegion,,1683694526936.48970235bef33aa5251ac68151d7d8b4.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-05-10 04:55:28,502 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=154, resume processing ppid=153 2023-05-10 04:55:28,502 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=154, ppid=153, state=SUCCESS; CloseRegionProcedure c5e0f2bc1f34b2b275d44dfbd485b943, server=jenkins-hbase4.apache.org,37823,1683694419534 in 176 msec 2023-05-10 04:55:28,503 DEBUG [StoreCloser-testSplitTableRegion,,1683694526936.48970235bef33aa5251ac68151d7d8b4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/48970235bef33aa5251ac68151d7d8b4/cf1/58d433c9ed254876ac1bfa1b33856169.e9be1c226b4184298a60d5e1076cd600 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegion/48970235bef33aa5251ac68151d7d8b4/cf1/58d433c9ed254876ac1bfa1b33856169.e9be1c226b4184298a60d5e1076cd600 2023-05-10 04:55:28,503 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=153, ppid=151, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=c5e0f2bc1f34b2b275d44dfbd485b943, UNASSIGN in 180 msec 2023-05-10 04:55:28,504 DEBUG [StoreCloser-testSplitTableRegion,,1683694526936.48970235bef33aa5251ac68151d7d8b4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/48970235bef33aa5251ac68151d7d8b4/cf1/34e676aeee2c4595a9dc599ec25aaf43.e9be1c226b4184298a60d5e1076cd600 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegion/48970235bef33aa5251ac68151d7d8b4/cf1/34e676aeee2c4595a9dc599ec25aaf43.e9be1c226b4184298a60d5e1076cd600 2023-05-10 04:55:28,505 DEBUG [StoreCloser-testSplitTableRegion,,1683694526936.48970235bef33aa5251ac68151d7d8b4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/48970235bef33aa5251ac68151d7d8b4/cf1/ec03059b17284629846b089825945254.e9be1c226b4184298a60d5e1076cd600 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegion/48970235bef33aa5251ac68151d7d8b4/cf1/ec03059b17284629846b089825945254.e9be1c226b4184298a60d5e1076cd600 2023-05-10 04:55:28,506 DEBUG [StoreCloser-testSplitTableRegion,,1683694526936.48970235bef33aa5251ac68151d7d8b4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/48970235bef33aa5251ac68151d7d8b4/cf1/91101e4c2e2e42578d390290caf782d5.e9be1c226b4184298a60d5e1076cd600 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegion/48970235bef33aa5251ac68151d7d8b4/cf1/91101e4c2e2e42578d390290caf782d5.e9be1c226b4184298a60d5e1076cd600 2023-05-10 04:55:28,506 DEBUG [StoreCloser-testSplitTableRegion,,1683694526936.48970235bef33aa5251ac68151d7d8b4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/48970235bef33aa5251ac68151d7d8b4/cf1/af98ce1ca36546c289891c252022ca3f.e9be1c226b4184298a60d5e1076cd600 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegion/48970235bef33aa5251ac68151d7d8b4/cf1/af98ce1ca36546c289891c252022ca3f.e9be1c226b4184298a60d5e1076cd600 2023-05-10 04:55:28,507 DEBUG [StoreCloser-testSplitTableRegion,,1683694526936.48970235bef33aa5251ac68151d7d8b4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/48970235bef33aa5251ac68151d7d8b4/cf1/46158d7d3eea4447ab610c5263cb7a20.e9be1c226b4184298a60d5e1076cd600 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegion/48970235bef33aa5251ac68151d7d8b4/cf1/46158d7d3eea4447ab610c5263cb7a20.e9be1c226b4184298a60d5e1076cd600 2023-05-10 04:55:28,508 DEBUG [StoreCloser-testSplitTableRegion,,1683694526936.48970235bef33aa5251ac68151d7d8b4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/48970235bef33aa5251ac68151d7d8b4/cf1/e7a24003dc044b82b9d96bae6d63b040.e9be1c226b4184298a60d5e1076cd600 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegion/48970235bef33aa5251ac68151d7d8b4/cf1/e7a24003dc044b82b9d96bae6d63b040.e9be1c226b4184298a60d5e1076cd600 2023-05-10 04:55:28,510 DEBUG [StoreCloser-testSplitTableRegion,,1683694526936.48970235bef33aa5251ac68151d7d8b4.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/48970235bef33aa5251ac68151d7d8b4/cf2/4664a731a30c46e2bf2e8bad410a89bd.e9be1c226b4184298a60d5e1076cd600->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf2/4664a731a30c46e2bf2e8bad410a89bd-bottom, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/48970235bef33aa5251ac68151d7d8b4/cf2/e29767cd1918445bbad6143229520f9e.e9be1c226b4184298a60d5e1076cd600->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf2/e29767cd1918445bbad6143229520f9e-bottom, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/48970235bef33aa5251ac68151d7d8b4/cf2/dd83d4fecabb4f31af903d8b1e8934b8.e9be1c226b4184298a60d5e1076cd600->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf2/dd83d4fecabb4f31af903d8b1e8934b8-bottom, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/48970235bef33aa5251ac68151d7d8b4/cf2/57e73fe7c40143d6a3a6e5618f59ed0a.e9be1c226b4184298a60d5e1076cd600->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf2/57e73fe7c40143d6a3a6e5618f59ed0a-bottom, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/48970235bef33aa5251ac68151d7d8b4/cf2/2cc0f196bb124e2eaab06768c81709b6.e9be1c226b4184298a60d5e1076cd600->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf2/2cc0f196bb124e2eaab06768c81709b6-bottom, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/48970235bef33aa5251ac68151d7d8b4/cf2/19631cfae8b94f0e9b5788c3a22fd03b.e9be1c226b4184298a60d5e1076cd600->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf2/19631cfae8b94f0e9b5788c3a22fd03b-bottom, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/48970235bef33aa5251ac68151d7d8b4/cf2/30a77dd573b64ac598ec88e3afa961e4.e9be1c226b4184298a60d5e1076cd600->hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf2/30a77dd573b64ac598ec88e3afa961e4-bottom] to archive 2023-05-10 04:55:28,510 DEBUG [StoreCloser-testSplitTableRegion,,1683694526936.48970235bef33aa5251ac68151d7d8b4.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-05-10 04:55:28,511 DEBUG [StoreCloser-testSplitTableRegion,,1683694526936.48970235bef33aa5251ac68151d7d8b4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/48970235bef33aa5251ac68151d7d8b4/cf2/4664a731a30c46e2bf2e8bad410a89bd.e9be1c226b4184298a60d5e1076cd600 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegion/48970235bef33aa5251ac68151d7d8b4/cf2/4664a731a30c46e2bf2e8bad410a89bd.e9be1c226b4184298a60d5e1076cd600 2023-05-10 04:55:28,512 DEBUG [StoreCloser-testSplitTableRegion,,1683694526936.48970235bef33aa5251ac68151d7d8b4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/48970235bef33aa5251ac68151d7d8b4/cf2/e29767cd1918445bbad6143229520f9e.e9be1c226b4184298a60d5e1076cd600 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegion/48970235bef33aa5251ac68151d7d8b4/cf2/e29767cd1918445bbad6143229520f9e.e9be1c226b4184298a60d5e1076cd600 2023-05-10 04:55:28,513 DEBUG [StoreCloser-testSplitTableRegion,,1683694526936.48970235bef33aa5251ac68151d7d8b4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/48970235bef33aa5251ac68151d7d8b4/cf2/dd83d4fecabb4f31af903d8b1e8934b8.e9be1c226b4184298a60d5e1076cd600 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegion/48970235bef33aa5251ac68151d7d8b4/cf2/dd83d4fecabb4f31af903d8b1e8934b8.e9be1c226b4184298a60d5e1076cd600 2023-05-10 04:55:28,514 DEBUG [StoreCloser-testSplitTableRegion,,1683694526936.48970235bef33aa5251ac68151d7d8b4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/48970235bef33aa5251ac68151d7d8b4/cf2/57e73fe7c40143d6a3a6e5618f59ed0a.e9be1c226b4184298a60d5e1076cd600 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegion/48970235bef33aa5251ac68151d7d8b4/cf2/57e73fe7c40143d6a3a6e5618f59ed0a.e9be1c226b4184298a60d5e1076cd600 2023-05-10 04:55:28,515 DEBUG [StoreCloser-testSplitTableRegion,,1683694526936.48970235bef33aa5251ac68151d7d8b4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/48970235bef33aa5251ac68151d7d8b4/cf2/2cc0f196bb124e2eaab06768c81709b6.e9be1c226b4184298a60d5e1076cd600 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegion/48970235bef33aa5251ac68151d7d8b4/cf2/2cc0f196bb124e2eaab06768c81709b6.e9be1c226b4184298a60d5e1076cd600 2023-05-10 04:55:28,516 DEBUG [StoreCloser-testSplitTableRegion,,1683694526936.48970235bef33aa5251ac68151d7d8b4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/48970235bef33aa5251ac68151d7d8b4/cf2/19631cfae8b94f0e9b5788c3a22fd03b.e9be1c226b4184298a60d5e1076cd600 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegion/48970235bef33aa5251ac68151d7d8b4/cf2/19631cfae8b94f0e9b5788c3a22fd03b.e9be1c226b4184298a60d5e1076cd600 2023-05-10 04:55:28,516 DEBUG [StoreCloser-testSplitTableRegion,,1683694526936.48970235bef33aa5251ac68151d7d8b4.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/48970235bef33aa5251ac68151d7d8b4/cf2/30a77dd573b64ac598ec88e3afa961e4.e9be1c226b4184298a60d5e1076cd600 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegion/48970235bef33aa5251ac68151d7d8b4/cf2/30a77dd573b64ac598ec88e3afa961e4.e9be1c226b4184298a60d5e1076cd600 2023-05-10 04:55:28,519 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/default/testSplitTableRegion/48970235bef33aa5251ac68151d7d8b4/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-05-10 04:55:28,520 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-10 04:55:28,520 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed testSplitTableRegion,,1683694526936.48970235bef33aa5251ac68151d7d8b4. 2023-05-10 04:55:28,520 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 48970235bef33aa5251ac68151d7d8b4: 2023-05-10 04:55:28,521 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.UnassignRegionHandler(149): Closed 48970235bef33aa5251ac68151d7d8b4 2023-05-10 04:55:28,521 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=152 updating hbase:meta row=48970235bef33aa5251ac68151d7d8b4, regionState=CLOSED 2023-05-10 04:55:28,522 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegion,,1683694526936.48970235bef33aa5251ac68151d7d8b4.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1683694528521"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1683694528521"}]},"ts":"1683694528521"} 2023-05-10 04:55:28,528 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=155, resume processing ppid=152 2023-05-10 04:55:28,528 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=155, ppid=152, state=SUCCESS; CloseRegionProcedure 48970235bef33aa5251ac68151d7d8b4, server=jenkins-hbase4.apache.org,37823,1683694419534 in 198 msec 2023-05-10 04:55:28,529 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=152, resume processing ppid=151 2023-05-10 04:55:28,529 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=152, ppid=151, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=48970235bef33aa5251ac68151d7d8b4, UNASSIGN in 205 msec 2023-05-10 04:55:28,530 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1683694528529"}]},"ts":"1683694528529"} 2023-05-10 04:55:28,530 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=DISABLED in hbase:meta 2023-05-10 04:55:28,532 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegion to state=DISABLED 2023-05-10 04:55:28,533 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=151, state=SUCCESS; DisableTableProcedure table=testSplitTableRegion in 217 msec 2023-05-10 04:55:28,621 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(1227): Checking to see if procedure is done pid=151 2023-05-10 04:55:28,621 INFO [Listener at localhost/45007] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegion, procId: 151 completed 2023-05-10 04:55:28,622 INFO [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.HMaster$5(2228): Client=jenkins//172.31.14.131 delete testSplitTableRegion 2023-05-10 04:55:28,622 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] procedure2.ProcedureExecutor(1029): Stored pid=156, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegion 2023-05-10 04:55:28,624 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=156, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-05-10 04:55:28,625 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(1227): Checking to see if procedure is done pid=156 2023-05-10 04:55:28,625 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=156, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-05-10 04:55:28,629 DEBUG [HFileArchiver-13] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600 2023-05-10 04:55:28,629 DEBUG [HFileArchiver-14] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegion/c5e0f2bc1f34b2b275d44dfbd485b943 2023-05-10 04:55:28,629 DEBUG [HFileArchiver-15] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegion/48970235bef33aa5251ac68151d7d8b4 2023-05-10 04:55:28,631 DEBUG [HFileArchiver-13] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf1, FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf2, FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/recovered.edits] 2023-05-10 04:55:28,631 DEBUG [HFileArchiver-15] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegion/48970235bef33aa5251ac68151d7d8b4/cf1, FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegion/48970235bef33aa5251ac68151d7d8b4/cf2, FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegion/48970235bef33aa5251ac68151d7d8b4/recovered.edits] 2023-05-10 04:55:28,631 DEBUG [HFileArchiver-14] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegion/c5e0f2bc1f34b2b275d44dfbd485b943/cf1, FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegion/c5e0f2bc1f34b2b275d44dfbd485b943/cf2, FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegion/c5e0f2bc1f34b2b275d44dfbd485b943/recovered.edits] 2023-05-10 04:55:28,634 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegion/48970235bef33aa5251ac68151d7d8b4/cf1/643c74205d064793bf31397cd72711db to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegion/48970235bef33aa5251ac68151d7d8b4/cf1/643c74205d064793bf31397cd72711db 2023-05-10 04:55:28,634 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegion/c5e0f2bc1f34b2b275d44dfbd485b943/cf1/557e0ebd8b0d4befa4afdf90e2f539b2 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegion/c5e0f2bc1f34b2b275d44dfbd485b943/cf1/557e0ebd8b0d4befa4afdf90e2f539b2 2023-05-10 04:55:28,635 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf1/34e676aeee2c4595a9dc599ec25aaf43 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf1/34e676aeee2c4595a9dc599ec25aaf43 2023-05-10 04:55:28,636 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf1/46158d7d3eea4447ab610c5263cb7a20 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf1/46158d7d3eea4447ab610c5263cb7a20 2023-05-10 04:55:28,637 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegion/c5e0f2bc1f34b2b275d44dfbd485b943/cf2/996e883a0cee4591ac08cfa7ae1e9935 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegion/c5e0f2bc1f34b2b275d44dfbd485b943/cf2/996e883a0cee4591ac08cfa7ae1e9935 2023-05-10 04:55:28,637 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegion/48970235bef33aa5251ac68151d7d8b4/cf2/bbb96d1f7d1b469788a061b27a1dc5ea to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegion/48970235bef33aa5251ac68151d7d8b4/cf2/bbb96d1f7d1b469788a061b27a1dc5ea 2023-05-10 04:55:28,638 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf1/58d433c9ed254876ac1bfa1b33856169 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf1/58d433c9ed254876ac1bfa1b33856169 2023-05-10 04:55:28,639 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf1/91101e4c2e2e42578d390290caf782d5 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf1/91101e4c2e2e42578d390290caf782d5 2023-05-10 04:55:28,640 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegion/48970235bef33aa5251ac68151d7d8b4/recovered.edits/90.seqid to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegion/48970235bef33aa5251ac68151d7d8b4/recovered.edits/90.seqid 2023-05-10 04:55:28,640 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegion/c5e0f2bc1f34b2b275d44dfbd485b943/recovered.edits/90.seqid to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegion/c5e0f2bc1f34b2b275d44dfbd485b943/recovered.edits/90.seqid 2023-05-10 04:55:28,641 DEBUG [HFileArchiver-15] backup.HFileArchiver(596): Deleted hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegion/48970235bef33aa5251ac68151d7d8b4 2023-05-10 04:55:28,641 DEBUG [HFileArchiver-14] backup.HFileArchiver(596): Deleted hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegion/c5e0f2bc1f34b2b275d44dfbd485b943 2023-05-10 04:55:28,641 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf1/af98ce1ca36546c289891c252022ca3f to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf1/af98ce1ca36546c289891c252022ca3f 2023-05-10 04:55:28,643 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf1/e7a24003dc044b82b9d96bae6d63b040 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf1/e7a24003dc044b82b9d96bae6d63b040 2023-05-10 04:55:28,644 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf1/ec03059b17284629846b089825945254 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf1/ec03059b17284629846b089825945254 2023-05-10 04:55:28,646 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf2/19631cfae8b94f0e9b5788c3a22fd03b to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf2/19631cfae8b94f0e9b5788c3a22fd03b 2023-05-10 04:55:28,647 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf2/2cc0f196bb124e2eaab06768c81709b6 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf2/2cc0f196bb124e2eaab06768c81709b6 2023-05-10 04:55:28,648 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf2/30a77dd573b64ac598ec88e3afa961e4 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf2/30a77dd573b64ac598ec88e3afa961e4 2023-05-10 04:55:28,649 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf2/4664a731a30c46e2bf2e8bad410a89bd to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf2/4664a731a30c46e2bf2e8bad410a89bd 2023-05-10 04:55:28,650 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf2/57e73fe7c40143d6a3a6e5618f59ed0a to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf2/57e73fe7c40143d6a3a6e5618f59ed0a 2023-05-10 04:55:28,651 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf2/dd83d4fecabb4f31af903d8b1e8934b8 to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf2/dd83d4fecabb4f31af903d8b1e8934b8 2023-05-10 04:55:28,652 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf2/e29767cd1918445bbad6143229520f9e to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/cf2/e29767cd1918445bbad6143229520f9e 2023-05-10 04:55:28,654 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/recovered.edits/85.seqid to hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/archive/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600/recovered.edits/85.seqid 2023-05-10 04:55:28,654 DEBUG [HFileArchiver-13] backup.HFileArchiver(596): Deleted hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/.tmp/data/default/testSplitTableRegion/e9be1c226b4184298a60d5e1076cd600 2023-05-10 04:55:28,655 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegion regions 2023-05-10 04:55:28,657 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=156, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-05-10 04:55:28,658 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegion from hbase:meta 2023-05-10 04:55:28,660 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegion' descriptor. 2023-05-10 04:55:28,661 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=156, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-05-10 04:55:28,661 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegion' from region states. 2023-05-10 04:55:28,661 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion,,1683694525376.e9be1c226b4184298a60d5e1076cd600.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1683694528661"}]},"ts":"9223372036854775807"} 2023-05-10 04:55:28,661 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion,,1683694526936.48970235bef33aa5251ac68151d7d8b4.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1683694528661"}]},"ts":"9223372036854775807"} 2023-05-10 04:55:28,661 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion,41,1683694526936.c5e0f2bc1f34b2b275d44dfbd485b943.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1683694528661"}]},"ts":"9223372036854775807"} 2023-05-10 04:55:28,662 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-05-10 04:55:28,663 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => e9be1c226b4184298a60d5e1076cd600, NAME => 'testSplitTableRegion,,1683694525376.e9be1c226b4184298a60d5e1076cd600.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 48970235bef33aa5251ac68151d7d8b4, NAME => 'testSplitTableRegion,,1683694526936.48970235bef33aa5251ac68151d7d8b4.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => c5e0f2bc1f34b2b275d44dfbd485b943, NAME => 'testSplitTableRegion,41,1683694526936.c5e0f2bc1f34b2b275d44dfbd485b943.', STARTKEY => '41', ENDKEY => ''}] 2023-05-10 04:55:28,663 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegion' as deleted. 2023-05-10 04:55:28,663 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1683694528663"}]},"ts":"9223372036854775807"} 2023-05-10 04:55:28,664 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegion state from META 2023-05-10 04:55:28,665 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=156, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-05-10 04:55:28,666 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=156, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegion in 43 msec 2023-05-10 04:55:28,726 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=37903] master.MasterRpcServices(1227): Checking to see if procedure is done pid=156 2023-05-10 04:55:28,726 INFO [Listener at localhost/45007] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegion, procId: 156 completed 2023-05-10 04:55:28,740 INFO [Listener at localhost/45007] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegion Thread=379 (was 377) - Thread LEAK? -, OpenFileDescriptor=841 (was 835) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=173 (was 173), ProcessCount=174 (was 174), AvailableMemoryMB=9473 (was 9531) 2023-05-10 04:55:28,740 INFO [Listener at localhost/45007] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-05-10 04:55:28,740 INFO [Listener at localhost/45007] client.ConnectionImplementation(1974): Closing master protocol: MasterService 2023-05-10 04:55:28,741 DEBUG [Listener at localhost/45007] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x0bbebf78 to 127.0.0.1:51177 2023-05-10 04:55:28,741 DEBUG [Listener at localhost/45007] ipc.AbstractRpcClient(494): Stopping rpc client 2023-05-10 04:55:28,742 DEBUG [Listener at localhost/45007] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-05-10 04:55:28,742 DEBUG [Listener at localhost/45007] util.JVMClusterUtil(257): Found active master hash=986602447, stopped=false 2023-05-10 04:55:28,742 INFO [Listener at localhost/45007] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase4.apache.org,37903,1683694418326 2023-05-10 04:55:28,932 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-05-10 04:55:28,932 INFO [Listener at localhost/45007] procedure2.ProcedureExecutor(629): Stopping 2023-05-10 04:55:28,932 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-05-10 04:55:28,932 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-05-10 04:55:28,932 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-05-10 04:55:28,933 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-05-10 04:55:28,933 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-05-10 04:55:28,933 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-05-10 04:55:28,933 DEBUG [Listener at localhost/45007] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x28c3bf4b to 127.0.0.1:51177 2023-05-10 04:55:28,934 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-05-10 04:55:28,933 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-05-10 04:55:28,934 DEBUG [Listener at localhost/45007] ipc.AbstractRpcClient(494): Stopping rpc client 2023-05-10 04:55:28,934 INFO [Listener at localhost/45007] regionserver.HRegionServer(2295): ***** STOPPING region server 'jenkins-hbase4.apache.org,36665,1683694419452' ***** 2023-05-10 04:55:28,934 INFO [Listener at localhost/45007] regionserver.HRegionServer(2309): STOPPED: Shutdown requested 2023-05-10 04:55:28,934 INFO [Listener at localhost/45007] regionserver.HRegionServer(2295): ***** STOPPING region server 'jenkins-hbase4.apache.org,37995,1683694419499' ***** 2023-05-10 04:55:28,934 INFO [Listener at localhost/45007] regionserver.HRegionServer(2309): STOPPED: Shutdown requested 2023-05-10 04:55:28,934 INFO [Listener at localhost/45007] regionserver.HRegionServer(2295): ***** STOPPING region server 'jenkins-hbase4.apache.org,37823,1683694419534' ***** 2023-05-10 04:55:28,934 INFO [RS:0;jenkins-hbase4:36665] regionserver.HeapMemoryManager(220): Stopping 2023-05-10 04:55:28,934 INFO [RS:0;jenkins-hbase4:36665] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-05-10 04:55:28,935 INFO [RS:0;jenkins-hbase4:36665] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-05-10 04:55:28,935 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-05-10 04:55:28,934 INFO [Listener at localhost/45007] regionserver.HRegionServer(2309): STOPPED: Shutdown requested 2023-05-10 04:55:28,935 INFO [RS:0;jenkins-hbase4:36665] regionserver.HRegionServer(3303): Received CLOSE for ccfda81dc2f13ffc1429bef67d349028 2023-05-10 04:55:28,935 INFO [RS:2;jenkins-hbase4:37823] regionserver.HeapMemoryManager(220): Stopping 2023-05-10 04:55:28,935 INFO [RS:2;jenkins-hbase4:37823] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-05-10 04:55:28,935 INFO [RS:2;jenkins-hbase4:37823] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-05-10 04:55:28,935 INFO [RS:2;jenkins-hbase4:37823] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:28,935 INFO [RS:0;jenkins-hbase4:36665] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:28,936 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-05-10 04:55:28,936 DEBUG [RS:0;jenkins-hbase4:36665] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x48577ed6 to 127.0.0.1:51177 2023-05-10 04:55:28,936 DEBUG [RS:0;jenkins-hbase4:36665] ipc.AbstractRpcClient(494): Stopping rpc client 2023-05-10 04:55:28,934 INFO [RS:1;jenkins-hbase4:37995] regionserver.HeapMemoryManager(220): Stopping 2023-05-10 04:55:28,936 INFO [RS:0;jenkins-hbase4:36665] regionserver.HRegionServer(1474): Waiting on 1 regions to close 2023-05-10 04:55:28,936 INFO [RS:1;jenkins-hbase4:37995] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-05-10 04:55:28,936 DEBUG [RS:0;jenkins-hbase4:36665] regionserver.HRegionServer(1478): Online Regions={ccfda81dc2f13ffc1429bef67d349028=hbase:namespace,,1683694421516.ccfda81dc2f13ffc1429bef67d349028.} 2023-05-10 04:55:28,936 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-05-10 04:55:28,935 DEBUG [RS:2;jenkins-hbase4:37823] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x38eee696 to 127.0.0.1:51177 2023-05-10 04:55:28,936 INFO [RS:1;jenkins-hbase4:37995] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-05-10 04:55:28,936 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing ccfda81dc2f13ffc1429bef67d349028, disabling compactions & flushes 2023-05-10 04:55:28,936 INFO [RS:1;jenkins-hbase4:37995] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:28,936 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1683694421516.ccfda81dc2f13ffc1429bef67d349028. 2023-05-10 04:55:28,936 DEBUG [RS:2;jenkins-hbase4:37823] ipc.AbstractRpcClient(494): Stopping rpc client 2023-05-10 04:55:28,936 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1683694421516.ccfda81dc2f13ffc1429bef67d349028. 2023-05-10 04:55:28,936 DEBUG [RS:1;jenkins-hbase4:37995] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x2d3688e7 to 127.0.0.1:51177 2023-05-10 04:55:28,936 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1683694421516.ccfda81dc2f13ffc1429bef67d349028. after waiting 0 ms 2023-05-10 04:55:28,936 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1683694421516.ccfda81dc2f13ffc1429bef67d349028. 2023-05-10 04:55:28,937 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing ccfda81dc2f13ffc1429bef67d349028 1/1 column families, dataSize=78 B heapSize=488 B 2023-05-10 04:55:28,937 DEBUG [RS:0;jenkins-hbase4:36665] regionserver.HRegionServer(1504): Waiting on ccfda81dc2f13ffc1429bef67d349028 2023-05-10 04:55:28,936 INFO [RS:2;jenkins-hbase4:37823] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,37823,1683694419534; all regions closed. 2023-05-10 04:55:28,936 DEBUG [RS:1;jenkins-hbase4:37995] ipc.AbstractRpcClient(494): Stopping rpc client 2023-05-10 04:55:28,937 INFO [RS:1;jenkins-hbase4:37995] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-05-10 04:55:28,937 INFO [RS:1;jenkins-hbase4:37995] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-05-10 04:55:28,937 INFO [RS:1;jenkins-hbase4:37995] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-05-10 04:55:28,937 INFO [RS:1;jenkins-hbase4:37995] regionserver.HRegionServer(3303): Received CLOSE for 1588230740 2023-05-10 04:55:28,938 INFO [RS:1;jenkins-hbase4:37995] regionserver.HRegionServer(1474): Waiting on 1 regions to close 2023-05-10 04:55:28,938 DEBUG [RS:1;jenkins-hbase4:37995] regionserver.HRegionServer(1478): Online Regions={1588230740=hbase:meta,,1.1588230740} 2023-05-10 04:55:28,938 DEBUG [RS:1;jenkins-hbase4:37995] regionserver.HRegionServer(1504): Waiting on 1588230740 2023-05-10 04:55:28,938 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-05-10 04:55:28,938 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-05-10 04:55:28,938 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-05-10 04:55:28,938 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-05-10 04:55:28,938 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-05-10 04:55:28,938 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=79.40 KB heapSize=125.77 KB 2023-05-10 04:55:28,977 DEBUG [RS:2;jenkins-hbase4:37823] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/oldWALs 2023-05-10 04:55:28,977 INFO [RS:2;jenkins-hbase4:37823] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C37823%2C1683694419534:(num 1683694421041) 2023-05-10 04:55:28,977 DEBUG [RS:2;jenkins-hbase4:37823] ipc.AbstractRpcClient(494): Stopping rpc client 2023-05-10 04:55:28,977 INFO [RS:2;jenkins-hbase4:37823] regionserver.LeaseManager(133): Closed leases 2023-05-10 04:55:28,977 INFO [RS:2;jenkins-hbase4:37823] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-05-10 04:55:28,978 INFO [RS:2;jenkins-hbase4:37823] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-05-10 04:55:28,978 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-05-10 04:55:28,978 INFO [RS:2;jenkins-hbase4:37823] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-05-10 04:55:28,978 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=78 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/hbase/namespace/ccfda81dc2f13ffc1429bef67d349028/.tmp/info/284ad87bb83a40388f13daf2c48a99bd 2023-05-10 04:55:28,978 INFO [RS:2;jenkins-hbase4:37823] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-05-10 04:55:28,979 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=71.29 KB at sequenceid=210 (bloomFilter=false), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/hbase/meta/1588230740/.tmp/info/31c55b209c8e4268bb6e09f518b772b6 2023-05-10 04:55:28,980 INFO [RS:2;jenkins-hbase4:37823] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:37823 2023-05-10 04:55:28,983 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 31c55b209c8e4268bb6e09f518b772b6 2023-05-10 04:55:28,986 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/hbase/namespace/ccfda81dc2f13ffc1429bef67d349028/.tmp/info/284ad87bb83a40388f13daf2c48a99bd as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/hbase/namespace/ccfda81dc2f13ffc1429bef67d349028/info/284ad87bb83a40388f13daf2c48a99bd 2023-05-10 04:55:28,988 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-05-10 04:55:28,988 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:28,988 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:28,989 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-05-10 04:55:28,988 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,37823,1683694419534 2023-05-10 04:55:28,989 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-05-10 04:55:28,989 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-05-10 04:55:28,991 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,37823,1683694419534] 2023-05-10 04:55:28,991 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,37823,1683694419534; numProcessing=1 2023-05-10 04:55:28,993 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/hbase/namespace/ccfda81dc2f13ffc1429bef67d349028/info/284ad87bb83a40388f13daf2c48a99bd, entries=2, sequenceid=6, filesize=4.8 K 2023-05-10 04:55:28,993 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,37823,1683694419534 already deleted, retry=false 2023-05-10 04:55:28,993 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,37823,1683694419534 expired; onlineServers=2 2023-05-10 04:55:28,994 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for ccfda81dc2f13ffc1429bef67d349028 in 57ms, sequenceid=6, compaction requested=false 2023-05-10 04:55:29,005 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.58 KB at sequenceid=210 (bloomFilter=false), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/hbase/meta/1588230740/.tmp/rep_barrier/e47f4a530e2c40818aa1ba8dde4f5041 2023-05-10 04:55:29,006 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/hbase/namespace/ccfda81dc2f13ffc1429bef67d349028/recovered.edits/9.seqid, newMaxSeqId=9, maxSeqId=1 2023-05-10 04:55:29,007 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-10 04:55:29,008 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1683694421516.ccfda81dc2f13ffc1429bef67d349028. 2023-05-10 04:55:29,008 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for ccfda81dc2f13ffc1429bef67d349028: 2023-05-10 04:55:29,008 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1683694421516.ccfda81dc2f13ffc1429bef67d349028. 2023-05-10 04:55:29,009 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e47f4a530e2c40818aa1ba8dde4f5041 2023-05-10 04:55:29,022 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=5.53 KB at sequenceid=210 (bloomFilter=false), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/hbase/meta/1588230740/.tmp/table/847b7cb31fe044e2a6c8ce99f671cbef 2023-05-10 04:55:29,024 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 847b7cb31fe044e2a6c8ce99f671cbef 2023-05-10 04:55:29,026 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/hbase/meta/1588230740/.tmp/info/31c55b209c8e4268bb6e09f518b772b6 as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/hbase/meta/1588230740/info/31c55b209c8e4268bb6e09f518b772b6 2023-05-10 04:55:29,029 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 31c55b209c8e4268bb6e09f518b772b6 2023-05-10 04:55:29,030 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/hbase/meta/1588230740/info/31c55b209c8e4268bb6e09f518b772b6, entries=59, sequenceid=210, filesize=11.1 K 2023-05-10 04:55:29,030 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/hbase/meta/1588230740/.tmp/rep_barrier/e47f4a530e2c40818aa1ba8dde4f5041 as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/hbase/meta/1588230740/rep_barrier/e47f4a530e2c40818aa1ba8dde4f5041 2023-05-10 04:55:29,031 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-05-10 04:55:29,031 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-05-10 04:55:29,031 INFO [regionserver/jenkins-hbase4:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-05-10 04:55:29,033 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for e47f4a530e2c40818aa1ba8dde4f5041 2023-05-10 04:55:29,033 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/hbase/meta/1588230740/rep_barrier/e47f4a530e2c40818aa1ba8dde4f5041, entries=24, sequenceid=210, filesize=7.5 K 2023-05-10 04:55:29,034 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/hbase/meta/1588230740/.tmp/table/847b7cb31fe044e2a6c8ce99f671cbef as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/hbase/meta/1588230740/table/847b7cb31fe044e2a6c8ce99f671cbef 2023-05-10 04:55:29,036 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 847b7cb31fe044e2a6c8ce99f671cbef 2023-05-10 04:55:29,036 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/hbase/meta/1588230740/table/847b7cb31fe044e2a6c8ce99f671cbef, entries=36, sequenceid=210, filesize=8.0 K 2023-05-10 04:55:29,037 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~79.40 KB/81308, heapSize ~125.73 KB/128744, currentSize=0 B/0 for 1588230740 in 99ms, sequenceid=210, compaction requested=false 2023-05-10 04:55:29,042 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/data/hbase/meta/1588230740/recovered.edits/213.seqid, newMaxSeqId=213, maxSeqId=1 2023-05-10 04:55:29,042 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-05-10 04:55:29,042 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-05-10 04:55:29,043 INFO [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-05-10 04:55:29,043 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-05-10 04:55:29,043 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase4:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-05-10 04:55:29,092 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-05-10 04:55:29,092 INFO [RS:2;jenkins-hbase4:37823] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,37823,1683694419534; zookeeper connection closed. 2023-05-10 04:55:29,092 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37823-0x1001370791d0003, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-05-10 04:55:29,092 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@532c3c7a] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@532c3c7a 2023-05-10 04:55:29,137 INFO [RS:0;jenkins-hbase4:36665] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,36665,1683694419452; all regions closed. 2023-05-10 04:55:29,138 INFO [RS:1;jenkins-hbase4:37995] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,37995,1683694419499; all regions closed. 2023-05-10 04:55:29,145 DEBUG [RS:0;jenkins-hbase4:36665] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/oldWALs 2023-05-10 04:55:29,145 INFO [RS:0;jenkins-hbase4:36665] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C36665%2C1683694419452:(num 1683694421041) 2023-05-10 04:55:29,145 DEBUG [RS:0;jenkins-hbase4:36665] ipc.AbstractRpcClient(494): Stopping rpc client 2023-05-10 04:55:29,145 INFO [RS:0;jenkins-hbase4:36665] regionserver.LeaseManager(133): Closed leases 2023-05-10 04:55:29,145 INFO [RS:0;jenkins-hbase4:36665] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-05-10 04:55:29,145 DEBUG [RS:1;jenkins-hbase4:37995] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/oldWALs 2023-05-10 04:55:29,146 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-05-10 04:55:29,146 INFO [RS:0;jenkins-hbase4:36665] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-05-10 04:55:29,146 INFO [RS:0;jenkins-hbase4:36665] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-05-10 04:55:29,146 INFO [RS:0;jenkins-hbase4:36665] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-05-10 04:55:29,146 INFO [RS:1;jenkins-hbase4:37995] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C37995%2C1683694419499.meta:.meta(num 1683694421250) 2023-05-10 04:55:29,147 INFO [RS:0;jenkins-hbase4:36665] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:36665 2023-05-10 04:55:29,148 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-05-10 04:55:29,148 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:29,148 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,36665,1683694419452 2023-05-10 04:55:29,151 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,36665,1683694419452] 2023-05-10 04:55:29,151 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,36665,1683694419452; numProcessing=2 2023-05-10 04:55:29,153 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,36665,1683694419452 already deleted, retry=false 2023-05-10 04:55:29,153 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,36665,1683694419452 expired; onlineServers=1 2023-05-10 04:55:29,154 DEBUG [RS:1;jenkins-hbase4:37995] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/oldWALs 2023-05-10 04:55:29,154 INFO [RS:1;jenkins-hbase4:37995] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase4.apache.org%2C37995%2C1683694419499:(num 1683694421042) 2023-05-10 04:55:29,154 DEBUG [RS:1;jenkins-hbase4:37995] ipc.AbstractRpcClient(494): Stopping rpc client 2023-05-10 04:55:29,154 INFO [RS:1;jenkins-hbase4:37995] regionserver.LeaseManager(133): Closed leases 2023-05-10 04:55:29,155 INFO [RS:1;jenkins-hbase4:37995] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase4:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-05-10 04:55:29,155 INFO [regionserver/jenkins-hbase4:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-05-10 04:55:29,156 INFO [RS:1;jenkins-hbase4:37995] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:37995 2023-05-10 04:55:29,157 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase4.apache.org,37995,1683694419499 2023-05-10 04:55:29,157 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-05-10 04:55:29,158 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase4.apache.org,37995,1683694419499] 2023-05-10 04:55:29,158 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase4.apache.org,37995,1683694419499; numProcessing=3 2023-05-10 04:55:29,160 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase4.apache.org,37995,1683694419499 already deleted, retry=false 2023-05-10 04:55:29,160 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase4.apache.org,37995,1683694419499 expired; onlineServers=0 2023-05-10 04:55:29,160 INFO [RegionServerTracker-0] regionserver.HRegionServer(2295): ***** STOPPING region server 'jenkins-hbase4.apache.org,37903,1683694418326' ***** 2023-05-10 04:55:29,160 INFO [RegionServerTracker-0] regionserver.HRegionServer(2309): STOPPED: Cluster shutdown set; onlineServer=0 2023-05-10 04:55:29,161 DEBUG [M:0;jenkins-hbase4:37903] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@7bf351f4, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase4.apache.org/172.31.14.131:0 2023-05-10 04:55:29,161 INFO [M:0;jenkins-hbase4:37903] regionserver.HRegionServer(1144): stopping server jenkins-hbase4.apache.org,37903,1683694418326 2023-05-10 04:55:29,161 INFO [M:0;jenkins-hbase4:37903] regionserver.HRegionServer(1170): stopping server jenkins-hbase4.apache.org,37903,1683694418326; all regions closed. 2023-05-10 04:55:29,161 DEBUG [M:0;jenkins-hbase4:37903] ipc.AbstractRpcClient(494): Stopping rpc client 2023-05-10 04:55:29,161 DEBUG [M:0;jenkins-hbase4:37903] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-05-10 04:55:29,161 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-05-10 04:55:29,161 DEBUG [M:0;jenkins-hbase4:37903] cleaner.HFileCleaner(317): Stopping file delete threads 2023-05-10 04:55:29,161 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1683694420697] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.large.0-1683694420697,5,FailOnTimeoutGroup] 2023-05-10 04:55:29,161 INFO [M:0;jenkins-hbase4:37903] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-05-10 04:55:29,161 INFO [M:0;jenkins-hbase4:37903] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-05-10 04:55:29,161 DEBUG [master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1683694420697] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase4:0:becomeActiveMaster-HFileCleaner.small.0-1683694420697,5,FailOnTimeoutGroup] 2023-05-10 04:55:29,161 INFO [M:0;jenkins-hbase4:37903] hbase.ChoreService(369): Chore service for: master/jenkins-hbase4:0 had [] on shutdown 2023-05-10 04:55:29,162 DEBUG [M:0;jenkins-hbase4:37903] master.HMaster(1512): Stopping service threads 2023-05-10 04:55:29,162 INFO [M:0;jenkins-hbase4:37903] procedure2.RemoteProcedureDispatcher(118): Stopping procedure remote dispatcher 2023-05-10 04:55:29,162 ERROR [M:0;jenkins-hbase4:37903] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-13,5,PEWorkerGroup] Thread[HFileArchiver-14,5,PEWorkerGroup] Thread[HFileArchiver-15,5,PEWorkerGroup] Thread[HFileArchiver-16,5,PEWorkerGroup] 2023-05-10 04:55:29,162 INFO [M:0;jenkins-hbase4:37903] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-05-10 04:55:29,162 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-05-10 04:55:29,162 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-05-10 04:55:29,162 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-05-10 04:55:29,162 DEBUG [M:0;jenkins-hbase4:37903] zookeeper.ZKUtil(398): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Unable to get data of znode /hbase/master because node does not exist (not an error) 2023-05-10 04:55:29,163 WARN [M:0;jenkins-hbase4:37903] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-05-10 04:55:29,163 INFO [M:0;jenkins-hbase4:37903] assignment.AssignmentManager(315): Stopping assignment manager 2023-05-10 04:55:29,163 INFO [M:0;jenkins-hbase4:37903] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-05-10 04:55:29,163 DEBUG [M:0;jenkins-hbase4:37903] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-05-10 04:55:29,163 INFO [M:0;jenkins-hbase4:37903] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-05-10 04:55:29,163 DEBUG [M:0;jenkins-hbase4:37903] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-05-10 04:55:29,163 DEBUG [M:0;jenkins-hbase4:37903] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-05-10 04:55:29,163 DEBUG [M:0;jenkins-hbase4:37903] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-05-10 04:55:29,163 INFO [M:0;jenkins-hbase4:37903] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=607.96 KB heapSize=718.95 KB 2023-05-10 04:55:29,166 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-05-10 04:55:29,177 INFO [M:0;jenkins-hbase4:37903] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=607.96 KB at sequenceid=1282 (bloomFilter=true), to=hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/6e602b76dc794c75b34d44af498e9caf 2023-05-10 04:55:29,180 INFO [M:0;jenkins-hbase4:37903] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6e602b76dc794c75b34d44af498e9caf 2023-05-10 04:55:29,180 DEBUG [M:0;jenkins-hbase4:37903] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/6e602b76dc794c75b34d44af498e9caf as hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/6e602b76dc794c75b34d44af498e9caf 2023-05-10 04:55:29,183 INFO [M:0;jenkins-hbase4:37903] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 6e602b76dc794c75b34d44af498e9caf 2023-05-10 04:55:29,183 INFO [M:0;jenkins-hbase4:37903] regionserver.HStore(1080): Added hdfs://localhost:38753/user/jenkins/test-data/b7b7042d-5744-ef3e-f29e-0686864034f7/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/6e602b76dc794c75b34d44af498e9caf, entries=156, sequenceid=1282, filesize=34.9 K 2023-05-10 04:55:29,184 INFO [M:0;jenkins-hbase4:37903] regionserver.HRegion(2948): Finished flush of dataSize ~607.96 KB/622556, heapSize ~718.93 KB/736184, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 21ms, sequenceid=1282, compaction requested=false 2023-05-10 04:55:29,184 INFO [M:0;jenkins-hbase4:37903] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-05-10 04:55:29,185 DEBUG [M:0;jenkins-hbase4:37903] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-05-10 04:55:29,189 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-05-10 04:55:29,189 INFO [M:0;jenkins-hbase4:37903] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-05-10 04:55:29,189 INFO [M:0;jenkins-hbase4:37903] ipc.NettyRpcServer(158): Stopping server on /172.31.14.131:37903 2023-05-10 04:55:29,190 DEBUG [M:0;jenkins-hbase4:37903] zookeeper.RecoverableZooKeeper(172): Node /hbase/rs/jenkins-hbase4.apache.org,37903,1683694418326 already deleted, retry=false 2023-05-10 04:55:29,253 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-05-10 04:55:29,253 INFO [RS:0;jenkins-hbase4:36665] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,36665,1683694419452; zookeeper connection closed. 2023-05-10 04:55:29,253 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:36665-0x1001370791d0001, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-05-10 04:55:29,253 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@32591f79] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@32591f79 2023-05-10 04:55:29,353 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-05-10 04:55:29,353 INFO [M:0;jenkins-hbase4:37903] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,37903,1683694418326; zookeeper connection closed. 2023-05-10 04:55:29,353 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): master:37903-0x1001370791d0000, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-05-10 04:55:29,453 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-05-10 04:55:29,453 DEBUG [Listener at localhost/45007-EventThread] zookeeper.ZKWatcher(600): regionserver:37995-0x1001370791d0002, quorum=127.0.0.1:51177, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-05-10 04:55:29,453 INFO [RS:1;jenkins-hbase4:37995] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase4.apache.org,37995,1683694419499; zookeeper connection closed. 2023-05-10 04:55:29,453 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@354c7ca3] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@354c7ca3 2023-05-10 04:55:29,454 INFO [Listener at localhost/45007] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 3 regionserver(s) complete 2023-05-10 04:55:29,454 WARN [Listener at localhost/45007] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-05-10 04:55:29,458 INFO [Listener at localhost/45007] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-05-10 04:55:29,564 WARN [BP-726857689-172.31.14.131-1683694414559 heartbeating to localhost/127.0.0.1:38753] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-05-10 04:55:29,566 WARN [BP-726857689-172.31.14.131-1683694414559 heartbeating to localhost/127.0.0.1:38753] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-726857689-172.31.14.131-1683694414559 (Datanode Uuid 0bc2fe53-c3db-437e-a8da-2e5e8eeaaf20) service to localhost/127.0.0.1:38753 2023-05-10 04:55:29,568 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5d73dea0-99f3-3103-481a-a34861abc6a4/cluster_105d7649-9bd1-85f0-42ad-07d3b5dd11a5/dfs/data/data5/current/BP-726857689-172.31.14.131-1683694414559] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-05-10 04:55:29,569 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5d73dea0-99f3-3103-481a-a34861abc6a4/cluster_105d7649-9bd1-85f0-42ad-07d3b5dd11a5/dfs/data/data6/current/BP-726857689-172.31.14.131-1683694414559] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-05-10 04:55:29,570 WARN [Listener at localhost/45007] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-05-10 04:55:29,574 INFO [Listener at localhost/45007] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-05-10 04:55:29,675 WARN [BP-726857689-172.31.14.131-1683694414559 heartbeating to localhost/127.0.0.1:38753] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-726857689-172.31.14.131-1683694414559 (Datanode Uuid a7b70f5c-571f-439d-b854-0d142c7d1170) service to localhost/127.0.0.1:38753 2023-05-10 04:55:29,676 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5d73dea0-99f3-3103-481a-a34861abc6a4/cluster_105d7649-9bd1-85f0-42ad-07d3b5dd11a5/dfs/data/data3/current/BP-726857689-172.31.14.131-1683694414559] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-05-10 04:55:29,676 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5d73dea0-99f3-3103-481a-a34861abc6a4/cluster_105d7649-9bd1-85f0-42ad-07d3b5dd11a5/dfs/data/data4/current/BP-726857689-172.31.14.131-1683694414559] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-05-10 04:55:29,678 WARN [Listener at localhost/45007] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-05-10 04:55:29,680 INFO [Listener at localhost/45007] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-05-10 04:55:29,746 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-05-10 04:55:29,794 WARN [BP-726857689-172.31.14.131-1683694414559 heartbeating to localhost/127.0.0.1:38753] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-05-10 04:55:29,794 WARN [BP-726857689-172.31.14.131-1683694414559 heartbeating to localhost/127.0.0.1:38753] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-726857689-172.31.14.131-1683694414559 (Datanode Uuid 87e3fd95-9de7-4a9c-ae21-87bf81b5e4b4) service to localhost/127.0.0.1:38753 2023-05-10 04:55:29,795 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5d73dea0-99f3-3103-481a-a34861abc6a4/cluster_105d7649-9bd1-85f0-42ad-07d3b5dd11a5/dfs/data/data1/current/BP-726857689-172.31.14.131-1683694414559] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-05-10 04:55:29,796 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/5d73dea0-99f3-3103-481a-a34861abc6a4/cluster_105d7649-9bd1-85f0-42ad-07d3b5dd11a5/dfs/data/data2/current/BP-726857689-172.31.14.131-1683694414559] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-05-10 04:55:29,819 INFO [Listener at localhost/45007] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-05-10 04:55:30,008 INFO [Listener at localhost/45007] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-05-10 04:55:30,250 INFO [Listener at localhost/45007] hbase.HBaseTestingUtility(1293): Minicluster is down